-- Logs begin at Tue 2022-09-06 19:25:57 UTC, end at Tue 2022-09-06 20:35:48 UTC. -- Sep 06 19:57:08.891526 dv-cinder-847732-4-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 06 19:57:08.900137 dv-cinder-847732-4-iscsi devstack@keystone.service[81759]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 06 19:57:08.905095 dv-cinder-847732-4-iscsi devstack@keystone.service[81759]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Sep 6 19:57:08 2022] *** Sep 06 19:57:08.905095 dv-cinder-847732-4-iscsi devstack@keystone.service[81759]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 06 19:57:08.905095 dv-cinder-847732-4-iscsi devstack@keystone.service[81759]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 06 19:57:08.905095 dv-cinder-847732-4-iscsi devstack@keystone.service[81759]: nodename: dv-cinder-847732-4-iscsi Sep 06 19:57:08.905095 dv-cinder-847732-4-iscsi devstack@keystone.service[81759]: machine: x86_64 Sep 06 19:57:08.905095 dv-cinder-847732-4-iscsi devstack@keystone.service[81759]: clock source: unix Sep 06 19:57:08.905095 dv-cinder-847732-4-iscsi devstack@keystone.service[81759]: pcre jit disabled Sep 06 19:57:08.905095 dv-cinder-847732-4-iscsi devstack@keystone.service[81759]: detected number of CPU cores: 4 Sep 06 19:57:08.905095 dv-cinder-847732-4-iscsi devstack@keystone.service[81759]: current working directory: / Sep 06 19:57:08.905095 dv-cinder-847732-4-iscsi devstack@keystone.service[81759]: detected binary path: /usr/bin/uwsgi-core Sep 06 19:57:08.905095 dv-cinder-847732-4-iscsi devstack@keystone.service[81759]: your processes number limit is 47680 Sep 06 19:57:08.905095 dv-cinder-847732-4-iscsi devstack@keystone.service[81759]: your memory page size is 4096 bytes Sep 06 19:57:08.905095 dv-cinder-847732-4-iscsi devstack@keystone.service[81759]: detected max file descriptor number: 2048 Sep 06 19:57:08.906356 dv-cinder-847732-4-iscsi devstack@keystone.service[81759]: lock engine: pthread robust mutexes Sep 06 19:57:08.906356 dv-cinder-847732-4-iscsi devstack@keystone.service[81759]: thunder lock: enabled Sep 06 19:57:08.906356 dv-cinder-847732-4-iscsi devstack@keystone.service[81759]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 06 19:57:08.906356 dv-cinder-847732-4-iscsi devstack@keystone.service[81759]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 06 19:57:08.956279 dv-cinder-847732-4-iscsi devstack@keystone.service[81759]: Python main interpreter initialized at 0x55e5b46be9c0 Sep 06 19:57:08.956279 dv-cinder-847732-4-iscsi devstack@keystone.service[81759]: python threads support enabled Sep 06 19:57:08.956279 dv-cinder-847732-4-iscsi devstack@keystone.service[81759]: your server socket listen backlog is limited to 100 connections Sep 06 19:57:08.956279 dv-cinder-847732-4-iscsi devstack@keystone.service[81759]: your mercy for graceful operations on workers is 90 seconds Sep 06 19:57:08.956914 dv-cinder-847732-4-iscsi devstack@keystone.service[81759]: mapped 403077 bytes (393 KB) for 2 cores Sep 06 19:57:08.956914 dv-cinder-847732-4-iscsi devstack@keystone.service[81759]: *** Operational MODE: preforking *** Sep 06 19:57:08.957167 dv-cinder-847732-4-iscsi devstack@keystone.service[81759]: *** uWSGI is running in multiple interpreter mode *** Sep 06 19:57:08.957167 dv-cinder-847732-4-iscsi devstack@keystone.service[81759]: spawned uWSGI master process (pid: 81759) Sep 06 19:57:08.957437 dv-cinder-847732-4-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 06 19:57:08.961695 dv-cinder-847732-4-iscsi devstack@keystone.service[81759]: spawned uWSGI worker 1 (pid: 81771, cores: 1) Sep 06 19:57:08.961695 dv-cinder-847732-4-iscsi devstack@keystone.service[81759]: spawned uWSGI worker 2 (pid: 81772, cores: 1) Sep 06 19:57:08.961695 dv-cinder-847732-4-iscsi devstack@keystone.service[81759]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 06 19:57:13.758281 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 06 19:57:13.758281 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: from cryptography.utils import int_from_bytes Sep 06 19:57:13.759523 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 06 19:57:13.759523 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: from cryptography.utils import int_from_bytes Sep 06 19:57:13.779177 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 06 19:57:13.779177 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: from cryptography.utils import int_from_bytes Sep 06 19:57:13.780483 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 06 19:57:13.780483 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: from cryptography.utils import int_from_bytes Sep 06 19:57:13.815439 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 06 19:57:13.815674 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 06 19:57:13.815818 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] command line args: [] {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 06 19:57:13.816169 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 06 19:57:13.816428 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] ================================================================================ {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 06 19:57:13.816775 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] admin_token = **** {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 19:57:13.817097 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] config_dir = [] {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 19:57:13.817347 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 19:57:13.817599 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] config_source = [] {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 19:57:13.817890 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] debug = True {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 19:57:13.818160 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 19:57:13.820177 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 19:57:13.820177 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 19:57:13.820177 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 19:57:13.820177 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 19:57:13.821352 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] list_limit = None {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 19:57:13.821352 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] log_config_append = None {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 19:57:13.821352 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 19:57:13.821352 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] log_dir = None {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 19:57:13.821352 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] log_file = None {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 19:57:13.821352 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 19:57:13.822995 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 19:57:13.822995 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 19:57:13.822995 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 19:57:13.822995 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 19:57:13.822995 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 19:57:13.822995 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 19:57:13.825152 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 19:57:13.825152 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 19:57:13.825152 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 19:57:13.825152 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 19:57:13.825152 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 19:57:13.825152 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 19:57:13.826502 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 19:57:13.826502 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 19:57:13.826502 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.20/identity {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 19:57:13.826896 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] publish_errors = False {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 19:57:13.827098 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 19:57:13.827281 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 19:57:13.827504 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 19:57:13.827803 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 19:57:13.828087 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 19:57:13.828367 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] standard_threads = False {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 19:57:13.828642 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 19:57:13.828936 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 19:57:13.829181 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 19:57:13.829356 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] use_journal = False {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 19:57:13.829542 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] use_json = False {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 19:57:13.829801 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] use_stderr = False {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 19:57:13.830016 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] use_syslog = False {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 19:57:13.830295 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 19:57:13.830608 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.830892 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.831199 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.831484 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.831788 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.831960 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 06 19:57:13.832190 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.832500 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 06 19:57:13.832500 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] command line args: [] {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 06 19:57:13.832852 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.833056 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 06 19:57:13.833247 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.833446 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] ================================================================================ {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 06 19:57:13.833602 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.833828 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] admin_token = **** {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 19:57:13.834007 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.834007 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.834332 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] config_dir = [] {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 19:57:13.834332 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 19:57:13.834591 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.834760 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] config_source = [] {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 19:57:13.834760 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] debug = True {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 19:57:13.835120 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.835299 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 19:57:13.835522 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.835703 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 19:57:13.835703 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 19:57:13.836032 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.836222 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 19:57:13.836401 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.836617 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 19:57:13.836617 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] list_limit = None {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 19:57:13.836942 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.837133 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] log_config_append = None {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 19:57:13.837133 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 19:57:13.837481 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.837718 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] log_dir = None {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 19:57:13.837718 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] log_file = None {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 19:57:13.838142 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.838142 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.838471 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 19:57:13.838471 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 19:57:13.838812 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.839017 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 19:57:13.839182 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.839351 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 19:57:13.839351 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 19:57:13.839693 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.839902 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 19:57:13.839902 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 19:57:13.840128 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.840242 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 19:57:13.840353 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.840449 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 19:57:13.840561 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.840652 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 19:57:13.840750 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.840862 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 19:57:13.841037 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.841165 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 19:57:13.841165 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 19:57:13.841414 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.841414 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.841682 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 19:57:13.841881 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.842040 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 19:57:13.842195 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.842195 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.842460 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.20/identity {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 19:57:13.842460 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] publish_errors = False {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 19:57:13.842808 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.842985 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 19:57:13.843147 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.843314 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 19:57:13.843314 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 19:57:13.843650 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.843837 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 19:57:13.843837 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 19:57:13.844174 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.844364 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] standard_threads = False {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 19:57:13.844547 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.844727 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 19:57:13.845028 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.845207 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 19:57:13.845207 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 19:57:13.845586 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.845804 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] use_journal = False {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 19:57:13.845985 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.846189 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] use_json = False {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 19:57:13.846189 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] use_stderr = False {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 19:57:13.846545 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.846712 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] use_syslog = False {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 19:57:13.846864 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.847049 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 19:57:13.847049 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.847400 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.847564 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.847746 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.847924 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.848110 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.848298 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.848476 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.848657 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.848800 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.848933 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.848933 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.849270 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.849467 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.849760 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.849937 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.850093 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.850259 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.850456 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.850652 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.850812 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.850997 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.850997 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.851346 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.851553 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.851753 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.851936 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.852126 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.852317 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.852514 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.852704 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.852888 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.853082 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.853268 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.853466 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.853466 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.853975 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.854142 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.854309 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.854495 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.854693 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.854899 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.855079 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.855260 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.855933 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.856117 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.856294 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.856489 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.856667 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.856843 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.857018 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.857206 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.857391 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.857592 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.857790 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.857971 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.858147 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.858321 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.858484 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.858670 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.858670 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.858976 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.859159 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.859334 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.859334 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.859829 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.860003 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.860217 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.860217 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.860636 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.860814 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.861006 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.861188 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.861365 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.861528 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.861798 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.861984 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.861984 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.862355 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.862538 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.862727 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.862903 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.863082 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.863256 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.863450 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.863635 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.863635 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.863972 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.864156 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.864333 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.864526 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.864526 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.864868 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.865055 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.865055 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.865385 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.865584 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.865792 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.865983 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.865983 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.866300 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.866447 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.866632 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.866820 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.866820 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.867170 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.867344 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.867507 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.867692 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.867692 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.868043 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.868233 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.868435 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.868618 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.868618 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.868982 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.869147 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.869308 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.869499 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.869711 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.869923 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.870118 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.870303 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.870303 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.870607 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.870760 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.870951 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.871195 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.871195 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.871495 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.871614 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.871730 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.871959 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.871959 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.872246 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.872432 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.872598 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.872750 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.872750 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.873104 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.873317 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.873449 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.873693 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.873869 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.874026 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.874026 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.874376 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] auth.external = None {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.874565 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.874746 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.874935 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.875113 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.875270 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.875270 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.875585 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.875776 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.875967 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] auth.password = None {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.876155 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.876366 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] auth.token = None {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.876541 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.876695 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.876848 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.877038 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.877215 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.877215 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.877511 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.877714 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.877869 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.878020 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.878199 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.878406 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.878406 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.878766 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.878965 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.879145 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.879309 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.879462 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] credential.caching = True {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.879654 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.879654 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.879937 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.880103 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.880292 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.880484 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.880655 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.880820 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.881004 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.881194 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.881194 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.881572 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.881785 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.881964 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.882119 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.882290 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.882480 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] auth.external = None {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.882643 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.882778 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.882980 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.883151 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.883151 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.883452 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.883646 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] auth.password = None {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.883841 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.884055 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] auth.token = None {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.884055 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.884398 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.884556 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.884730 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.884912 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.885126 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.885312 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.885312 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.885581 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.885861 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.886015 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] federation.caching = True {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.886192 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.886396 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.886559 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] credential.caching = True {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.887177 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.887365 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.887544 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.887711 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.887711 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.888038 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.888220 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.888220 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.888570 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.888764 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.888923 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.889083 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.889083 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.889421 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.889635 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.889850 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.890046 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.890226 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.890382 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.890544 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.890714 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.890899 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.891073 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.891246 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] identity.caching = True {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.891438 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.891438 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.891790 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.891996 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] federation.caching = True {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.892189 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.892397 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.892585 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.892748 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.892912 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.893187 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.893365 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.893559 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.893559 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.893940 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.894134 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.894307 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.894477 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.894669 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.894843 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.895005 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.895178 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.895360 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.895552 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.895737 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.895927 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.895927 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] identity.caching = True {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.896293 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.896423 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.896608 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.896796 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.896954 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.897117 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.897294 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.897477 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.897477 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.897920 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.898119 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.898289 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.898475 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.898665 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.898845 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.898845 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.899133 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.899316 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.899513 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.899694 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.899879 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.900085 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.900231 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.900388 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.900581 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.900769 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.900769 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.901129 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.901294 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.901471 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.901697 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.901978 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.902134 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.902312 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.902541 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.902737 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.902940 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.902940 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.903256 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.903396 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.903573 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.903752 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.903983 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.904162 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.904331 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.904523 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.904658 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.904846 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.904846 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.905205 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.905375 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.905538 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.905735 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.905913 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.906054 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.906204 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.906402 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.906402 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.906700 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.906863 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.907069 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.907268 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.907447 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.907626 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.907806 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.907991 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.908151 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.908272 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.908272 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.908531 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.908704 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.908704 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.909048 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.909189 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.909361 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.909549 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.909762 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.909951 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.909951 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.910578 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.910772 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.910961 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.911149 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.911337 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.911544 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.911736 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.911923 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.911923 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.912277 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] ldap.user = None {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.912277 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.912649 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.912649 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] ldap.user = None {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.912998 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.913181 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.913353 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.913530 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.913813 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.914002 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.914002 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.914360 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.914551 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.914742 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.914932 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.915114 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.915296 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.915481 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.915668 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.915668 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.916026 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.916201 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.916383 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.916571 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.916750 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.916948 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.917137 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.917333 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.917518 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.917726 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.917726 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.918590 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.918773 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.918961 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.919137 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.919316 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.919510 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.919510 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.919877 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.920071 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.920261 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.920446 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.920630 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.920837 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.920837 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.921198 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.921388 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.921584 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.921866 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.922083 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.922276 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.922276 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.922627 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.922827 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.923019 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.923223 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.923223 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.923577 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.923772 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.923772 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.924118 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.924312 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.924495 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.924685 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.924871 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.925061 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.925061 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.925443 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.925661 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] resource.caching = True {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.925870 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.926066 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.926251 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.926443 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.926443 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.926818 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.927000 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.927179 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.927357 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.927540 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.927738 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.927738 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.928081 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.928265 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.928449 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.928634 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.928958 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] resource.caching = True {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.929152 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] role.caching = True {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.929341 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.929518 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] role.driver = sql {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.929518 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.929987 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.930201 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.930392 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.930580 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.930748 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.930911 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.931090 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.931272 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.931272 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.931624 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.931806 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.931998 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.932236 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.932515 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.932705 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.932890 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.933070 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.933070 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.933421 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] role.caching = True {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.933421 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] role.driver = sql {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.933863 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.934048 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.934252 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.934252 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.934611 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.934611 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.934946 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.935131 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.935326 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.935516 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.935716 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.935905 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.936083 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.936083 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.936434 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.936621 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.936821 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.937008 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.937198 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.937382 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.937566 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.937770 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.937974 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.938157 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.938157 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.938501 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.938685 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.938868 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.939065 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.939247 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.939431 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.939431 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.939431 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.939931 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.940124 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.940309 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.940498 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.940681 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.940879 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.941066 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.941266 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] token.caching = True {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.941461 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.941701 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.941701 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.942110 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.942297 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.942297 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.942680 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.942871 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.943058 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.943249 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.943453 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.943650 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.943838 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.944024 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.944024 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.944379 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.944585 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.944781 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.944971 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.945158 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.945355 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.945355 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.945768 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.945958 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.946145 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.946334 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.946334 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.946724 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.946909 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.947090 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.947273 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] database.connection = **** {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.947449 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.947630 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.947810 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.947996 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.947996 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.948349 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] token.caching = True {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.948527 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.948716 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.948893 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.948893 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.949278 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.949464 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.949676 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.949865 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.949865 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.950216 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.950404 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.950600 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.950776 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.950936 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.951087 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.951248 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.951411 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.951411 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.951732 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.951918 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.952102 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.952289 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.952289 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.952654 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.952841 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.953027 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.953215 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81772) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 06 19:57:13.953403 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.953403 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.953403 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.953403 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.953403 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.953403 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.953403 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] database.connection = **** {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.953403 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.953403 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.954964 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.954964 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.954964 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.954964 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.954964 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.954964 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.954964 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.954964 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.954964 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.956511 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.956511 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.956511 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.956511 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.956511 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.956511 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.956511 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 19:57:13.956511 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81771) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 06 19:57:13.961503 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81772) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 19:57:13.961973 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81772) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 19:57:13.962175 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81772) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 19:57:13.967343 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81771) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 19:57:13.967574 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81771) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 19:57:13.967795 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81771) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 19:57:13.971795 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81772) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 19:57:13.977852 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81771) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 19:57:14.004221 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81772) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 19:57:14.004561 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81772) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 19:57:14.008976 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81771) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 19:57:14.009387 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81771) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 19:57:14.067156 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81771) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 19:57:14.083055 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81772) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 19:57:14.120707 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81771) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 19:57:14.120931 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81771) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 19:57:14.155681 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81772) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 19:57:14.156064 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81772) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 19:57:14.260935 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81772) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 19:57:14.261410 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81772) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 19:57:14.261673 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81772) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 19:57:14.261975 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81772) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 19:57:14.262331 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81772) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 19:57:14.262611 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81772) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 19:57:14.284037 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81771) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 19:57:14.284378 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81771) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 19:57:14.284685 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81771) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 19:57:14.284998 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81771) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 19:57:14.285301 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81771) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 19:57:14.285887 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81771) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 19:57:14.300811 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81771) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 19:57:14.301083 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81771) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 19:57:14.301332 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81771) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 19:57:14.301563 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81771) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 19:57:14.301853 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81771) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 19:57:14.302105 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81771) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 19:57:14.302319 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81771) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 19:57:14.316775 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81771) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 19:57:14.331212 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81771) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 19:57:14.331447 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81771) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 19:57:14.331581 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81771) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 19:57:14.331767 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81771) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 19:57:14.331990 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81771) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 19:57:14.332350 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81771) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 19:57:14.332502 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81771) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 19:57:14.351155 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81771) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.351417 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81771) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 06 19:57:14.351756 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81771) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.351957 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81771) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 06 19:57:14.352240 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81771) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.352509 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81771) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.352779 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81771) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.353047 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81771) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.363779 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81771) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.364084 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81771) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.364337 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81771) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.364626 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81771) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.372239 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81771) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 19:57:14.375520 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81771) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 19:57:14.375943 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81771) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.376308 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81771) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.376749 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81771) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.377120 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81771) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.377469 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81771) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.377864 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81771) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.378229 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81771) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.378622 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81771) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.378989 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81771) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.379348 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81771) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.386092 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81772) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 19:57:14.386382 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81772) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 19:57:14.386675 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81772) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 19:57:14.387481 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81772) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 19:57:14.387852 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81772) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 19:57:14.388038 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81772) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 19:57:14.388343 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81772) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 19:57:14.397248 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81772) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 19:57:14.400553 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81772) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 19:57:14.400844 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81772) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 19:57:14.401134 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81772) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 19:57:14.401455 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81772) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 19:57:14.401775 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81772) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 19:57:14.402061 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81772) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 19:57:14.402345 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81772) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 19:57:14.405884 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81771) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.407411 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81771) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 19:57:14.407696 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81771) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.412873 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81771) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 19:57:14.413275 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81771) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.413661 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81771) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.414081 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81772) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.414081 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81772) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 06 19:57:14.414566 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81772) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.414831 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81772) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 06 19:57:14.415177 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81772) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.415512 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81772) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.415839 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81772) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.416167 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81772) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.421933 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81771) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 19:57:14.422352 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81771) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.426888 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81772) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.427268 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81772) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.427612 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81772) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.427967 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81772) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.428938 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81771) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 19:57:14.429340 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81771) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.429761 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81771) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.430142 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81771) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.430501 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81771) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.430862 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81771) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.431217 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81771) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.431570 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81771) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.436205 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81772) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 19:57:14.439355 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81772) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 19:57:14.439716 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81772) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.440071 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81772) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.440422 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81772) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.440785 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81772) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.441124 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81772) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.441510 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81772) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.441927 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81772) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.442280 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81772) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.442768 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81772) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.443116 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81772) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.449282 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81771) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.450228 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81771) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.454707 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81771) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 19:57:14.458564 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81771) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.458958 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81771) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.463547 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81771) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 19:57:14.467076 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81771) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 19:57:14.469268 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81772) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.470802 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81771) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.471160 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81772) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 19:57:14.471416 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81771) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.471613 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81772) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.471888 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81771) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.472270 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81771) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.472627 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81771) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.473067 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81771) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.476946 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81772) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 19:57:14.477340 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81772) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.477747 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81772) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.485218 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81772) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 19:57:14.485653 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81772) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.490741 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81772) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 19:57:14.491169 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81772) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.491530 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81772) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.491677 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81771) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 19:57:14.491934 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81772) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.492080 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81771) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.492350 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81772) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.492494 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81771) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.492835 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81772) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.493028 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81771) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.493176 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81772) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.493604 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81772) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.500846 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81771) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.502624 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81771) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.504625 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81771) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.505007 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81771) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.507959 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81771) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 19:57:14.508349 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81771) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.508717 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81771) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.509080 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81771) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.509572 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81771) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.520544 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81772) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.520933 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81772) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.525145 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81771) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 19:57:14.525451 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81771) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.525859 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81771) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.528176 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81771) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.528803 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81772) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 19:57:14.528968 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81771) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.529252 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81771) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.529791 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81771) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.535070 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81772) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.535814 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81772) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.544334 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81772) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 19:57:14.550512 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81772) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 19:57:14.556455 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81772) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.559396 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81772) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.559396 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81772) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.559396 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81772) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.559396 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81772) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.559396 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81772) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.573366 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81771) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 19:57:14.578697 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81771) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 19:57:14.583561 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81771) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.586372 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81771) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.586605 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81772) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 19:57:14.586998 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81772) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.587355 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81772) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.587709 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81772) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.588832 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81771) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 19:57:14.589265 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81771) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.589906 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81771) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.595757 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81772) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.597473 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81772) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.599173 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81772) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.599551 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81772) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.601085 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81771) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.602804 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81772) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 19:57:14.603314 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81772) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.603675 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81771) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 19:57:14.603828 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81772) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.604193 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81772) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.604571 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81772) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.609219 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81771) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.609942 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81771) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.610279 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81771) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.610701 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81771) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.618073 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81772) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 19:57:14.618426 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81772) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.618858 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81772) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.619148 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81772) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.619434 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81772) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.619726 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81772) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.620015 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81772) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.622936 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81771) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 19:57:14.623403 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81771) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.623815 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81771) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.633033 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81771) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 19:57:14.633479 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81771) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.633921 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81771) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.634332 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81771) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.634736 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81771) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.635150 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81771) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.635560 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81771) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.635963 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81771) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.636375 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81771) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.637514 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81771) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.637866 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81771) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.638235 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81771) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.638606 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81771) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.639036 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81771) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.639311 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81772) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 19:57:14.643905 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81772) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 19:57:14.647202 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81772) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.648841 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81772) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.650663 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81772) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 19:57:14.651045 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81772) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.651414 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81772) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.659469 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81772) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.661243 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81772) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 19:57:14.664670 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81772) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.665112 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81772) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.665336 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81772) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.665662 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81772) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.674812 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81772) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 19:57:14.675212 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81772) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.675569 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81772) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.683768 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55e5b46be9c0 pid: 81771 (default app) Sep 06 19:57:14.684374 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81772) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 19:57:14.684777 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81772) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.685139 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81772) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.685490 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81772) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.685866 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81772) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.686250 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81772) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.686610 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81772) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.686961 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81772) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.687373 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81772) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.687694 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81772) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.687833 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8b77ee-24d9-4ed7-84a3-e708719f888d None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:14.688055 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8b77ee-24d9-4ed7-84a3-e708719f888d None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:14.688198 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81772) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.688353 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8b77ee-24d9-4ed7-84a3-e708719f888d None None] PATH_INFO: `/v3` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:14.688536 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81772) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.688895 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81772) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.689240 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81772) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 19:57:14.693859 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1/1] 10.222.0.20 () {50 vars in 791 bytes} [Tue Sep 6 19:57:14 2022] GET /identity/v3/ => generated 255 bytes in 9 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:57:14.726941 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55e5b46be9c0 pid: 81772 (default app) Sep 06 19:57:22.340834 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3049b980-4820-4d26-b302-edc1f8dafb3d None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:22.341756 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3049b980-4820-4d26-b302-edc1f8dafb3d None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:22.341756 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3049b980-4820-4d26-b302-edc1f8dafb3d None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:22.343286 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 2/2] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 19:57:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 19:57:22.355451 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efb5e65-3650-4ce4-b112-f9d176bda52c None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:22.356209 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efb5e65-3650-4ce4-b112-f9d176bda52c None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:22.356419 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efb5e65-3650-4ce4-b112-f9d176bda52c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:22.505999 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: INFO dbcounter [None req-2efb5e65-3650-4ce4-b112-f9d176bda52c None None] Registered counter for database keystone Sep 06 19:57:22.542360 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG oslo_db.sqlalchemy.engines [None req-2efb5e65-3650-4ce4-b112-f9d176bda52c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81772) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 06 19:57:22.554090 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writer thread running {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 06 19:57:22.919218 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG passlib.handlers.bcrypt [None req-2efb5e65-3650-4ce4-b112-f9d176bda52c None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81772) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 06 19:57:22.919608 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG passlib.handlers.bcrypt [None req-2efb5e65-3650-4ce4-b112-f9d176bda52c None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81772) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 06 19:57:22.984093 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-2efb5e65-3650-4ce4-b112-f9d176bda52c None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 19:57:22.987015 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-2efb5e65-3650-4ce4-b112-f9d176bda52c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 19:57:23.169840 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1/3] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 19:57:22 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 821 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Sep 06 19:57:23.861537 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3049b980-4820-4d26-b302-edc1f8dafb3d None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:57:23.944560 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: INFO dbcounter [None req-3049b980-4820-4d26-b302-edc1f8dafb3d None None] Registered counter for database keystone Sep 06 19:57:23.971665 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG oslo_db.sqlalchemy.engines [None req-3049b980-4820-4d26-b302-edc1f8dafb3d None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81771) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 06 19:57:23.982644 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writer thread running {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 06 19:57:24.187973 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3049b980-4820-4d26-b302-edc1f8dafb3d None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:57:24.192708 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb14a3a-ddc0-48d1-a13f-f1965fd4be77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:57:24.193431 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb14a3a-ddc0-48d1-a13f-f1965fd4be77 admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:24.193783 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb14a3a-ddc0-48d1-a13f-f1965fd4be77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:24.194092 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb14a3a-ddc0-48d1-a13f-f1965fd4be77 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:24.204861 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb14a3a-ddc0-48d1-a13f-f1965fd4be77 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:57:24.257791 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: WARNING keystone.common.rbac_enforcer.enforcer [None req-beb14a3a-ddc0-48d1-a13f-f1965fd4be77 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 06 19:57:24.258180 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb14a3a-ddc0-48d1-a13f-f1965fd4be77 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:57:24.270970 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 3/4] 10.222.0.20 () {58 vars in 1324 bytes} [Tue Sep 6 19:57:23 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 06 19:57:24.278603 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efb5e65-3650-4ce4-b112-f9d176bda52c None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:57:24.307314 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efb5e65-3650-4ce4-b112-f9d176bda52c None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:57:24.315677 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50ce370-01c8-4624-a76b-77b0b8d24364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:57:24.318342 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50ce370-01c8-4624-a76b-77b0b8d24364 admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:24.318865 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50ce370-01c8-4624-a76b-77b0b8d24364 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:24.319441 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50ce370-01c8-4624-a76b-77b0b8d24364 admin admin] PATH_INFO: `/v3/projects/a630b7748ecd4893b8db1b6c1dc4efe9` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:24.322692 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50ce370-01c8-4624-a76b-77b0b8d24364 admin admin] RBAC: Authorizing `identity:get_project(project_id=a630b7748ecd4893b8db1b6c1dc4efe9)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:57:24.440634 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: WARNING keystone.common.rbac_enforcer.enforcer [None req-e50ce370-01c8-4624-a76b-77b0b8d24364 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 06 19:57:24.441178 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50ce370-01c8-4624-a76b-77b0b8d24364 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:57:24.451680 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 2/5] 10.222.0.20 () {56 vars in 1207 bytes} [Tue Sep 6 19:57:24 2022] GET /identity/v3/projects/a630b7748ecd4893b8db1b6c1dc4efe9 => generated 344 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:57:26.782979 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054393e9-a89c-476b-a9fa-03efaa3917e1 None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:26.783643 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054393e9-a89c-476b-a9fa-03efaa3917e1 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:26.783643 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054393e9-a89c-476b-a9fa-03efaa3917e1 None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:26.788962 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 4/6] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 19:57:26 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 19:57:26.796514 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfc7b64-da51-4ad2-bf5c-b53718d35a41 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:26.797104 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfc7b64-da51-4ad2-bf5c-b53718d35a41 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:26.797237 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfc7b64-da51-4ad2-bf5c-b53718d35a41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:26.820967 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-2bfc7b64-da51-4ad2-bf5c-b53718d35a41 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 19:57:26.822503 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-2bfc7b64-da51-4ad2-bf5c-b53718d35a41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 19:57:26.831012 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 3/7] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 19:57:26 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 06 19:57:27.109941 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054393e9-a89c-476b-a9fa-03efaa3917e1 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:57:27.129397 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054393e9-a89c-476b-a9fa-03efaa3917e1 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:57:27.134284 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a5470b-ec43-43e6-b14e-9ee8a9d652e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:57:27.134986 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a5470b-ec43-43e6-b14e-9ee8a9d652e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:27.135360 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a5470b-ec43-43e6-b14e-9ee8a9d652e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:27.135649 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a5470b-ec43-43e6-b14e-9ee8a9d652e7 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:27.145483 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a5470b-ec43-43e6-b14e-9ee8a9d652e7 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:57:27.147407 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a5470b-ec43-43e6-b14e-9ee8a9d652e7 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:57:27.157998 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 5/8] 10.222.0.20 () {58 vars in 1324 bytes} [Tue Sep 6 19:57:27 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 19:57:27.165293 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfc7b64-da51-4ad2-bf5c-b53718d35a41 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:57:27.177666 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfc7b64-da51-4ad2-bf5c-b53718d35a41 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:57:27.185524 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4821199c-f1de-4b44-9c76-d7264a472231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:57:27.186300 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4821199c-f1de-4b44-9c76-d7264a472231 admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:27.186628 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4821199c-f1de-4b44-9c76-d7264a472231 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:27.186956 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4821199c-f1de-4b44-9c76-d7264a472231 admin admin] PATH_INFO: `/v3/users/1e7095a0137b45c984179dcf25de9362` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:27.188638 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4821199c-f1de-4b44-9c76-d7264a472231 admin admin] RBAC: Authorizing `identity:get_user(user_id=1e7095a0137b45c984179dcf25de9362)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:57:27.190760 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4821199c-f1de-4b44-9c76-d7264a472231 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:57:27.192560 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 4/9] 10.222.0.20 () {56 vars in 1198 bytes} [Tue Sep 6 19:57:27 2022] GET /identity/v3/users/1e7095a0137b45c984179dcf25de9362 => generated 247 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:57:29.666680 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274111d1-1fd2-40e9-a040-141b2c561a5c None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:29.667266 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274111d1-1fd2-40e9-a040-141b2c561a5c None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:29.667652 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274111d1-1fd2-40e9-a040-141b2c561a5c None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:29.670288 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 6/10] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 19:57:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 19:57:29.680401 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cb6986-6994-4cad-98f0-222c5a786363 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:29.681269 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cb6986-6994-4cad-98f0-222c5a786363 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:29.681269 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cb6986-6994-4cad-98f0-222c5a786363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:29.704005 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-18cb6986-6994-4cad-98f0-222c5a786363 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 19:57:29.705742 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-18cb6986-6994-4cad-98f0-222c5a786363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 19:57:29.829089 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 5/11] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 19:57:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 19:57:29.829485 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35229298-8d9b-4fc0-8163-963f24d75279 None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:29.829689 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35229298-8d9b-4fc0-8163-963f24d75279 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:29.829930 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35229298-8d9b-4fc0-8163-963f24d75279 None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:29.831384 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 7/12] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 19:57:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 19:57:29.838317 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed905209-2ef6-438f-a142-3ad1e5ab6721 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:29.838675 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed905209-2ef6-438f-a142-3ad1e5ab6721 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:29.839018 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed905209-2ef6-438f-a142-3ad1e5ab6721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:29.869996 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-ed905209-2ef6-438f-a142-3ad1e5ab6721 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 19:57:29.871824 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-ed905209-2ef6-438f-a142-3ad1e5ab6721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 19:57:29.899252 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:57:29.982528 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 6/13] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 19:57:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 19:57:30.131979 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35229298-8d9b-4fc0-8163-963f24d75279 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:57:30.280174 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed905209-2ef6-438f-a142-3ad1e5ab6721 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:57:30.327582 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35229298-8d9b-4fc0-8163-963f24d75279 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:57:30.387973 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd88825e-de90-4eb3-8175-c9285e07248a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:57:30.388963 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd88825e-de90-4eb3-8175-c9285e07248a None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:30.389466 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd88825e-de90-4eb3-8175-c9285e07248a None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:30.390007 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd88825e-de90-4eb3-8175-c9285e07248a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:30.435407 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed905209-2ef6-438f-a142-3ad1e5ab6721 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:57:30.471962 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:57:30.506135 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7470299-c699-4e7b-aa5c-f858f9e6b79a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:57:30.507266 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7470299-c699-4e7b-aa5c-f858f9e6b79a None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:30.507756 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7470299-c699-4e7b-aa5c-f858f9e6b79a None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:30.508269 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7470299-c699-4e7b-aa5c-f858f9e6b79a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:30.510817 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd88825e-de90-4eb3-8175-c9285e07248a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:57:30.512351 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd88825e-de90-4eb3-8175-c9285e07248a None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:57:30.532841 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7470299-c699-4e7b-aa5c-f858f9e6b79a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 19:57:30.551888 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-c7470299-c699-4e7b-aa5c-f858f9e6b79a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 19:57:30.551888 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:57:30.551888 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:57:30.551888 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:57:30.551888 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:57:30.551888 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:57:30.551888 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:57:30.551888 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:57:30.551888 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:57:30.551888 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:57:30.551888 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:57:30.551888 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:57:30.551888 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 19:57:30.551888 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 19:57:30.551888 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 19:57:30.551888 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 19:57:30.553212 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 19:57:30.553212 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 19:57:30.553212 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 19:57:30.553212 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 19:57:30.553212 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:57:30.553212 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:57:30.553212 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:57:30.553212 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:57:30.553212 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:57:30.553212 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:57:30.553212 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:57:30.553212 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 19:57:30.553212 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:57:30.553212 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:57:30.553212 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:57:30.553212 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:57:30.554753 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:57:30.554753 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:57:30.554753 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:57:30.554753 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:57:30.554753 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 19:57:30.554753 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 19:57:30.554753 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 19:57:30.554753 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 19:57:30.554753 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 19:57:30.554753 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 19:57:30.554753 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 19:57:30.554753 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 19:57:30.554753 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 7/14] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Sep 6 19:57:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 19:57:30.564151 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7470299-c699-4e7b-aa5c-f858f9e6b79a None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:57:30.592561 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:57:30.651357 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 8/15] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 6 19:57:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 520 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 06 19:57:30.659181 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd88825e-de90-4eb3-8175-c9285e07248a None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:57:30.721541 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7470299-c699-4e7b-aa5c-f858f9e6b79a None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:57:30.757441 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:57:30.784953 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e111189d-0eb8-4b19-acf4-91791e90fae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:57:30.785581 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e111189d-0eb8-4b19-acf4-91791e90fae6 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:30.786001 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e111189d-0eb8-4b19-acf4-91791e90fae6 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:30.786406 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e111189d-0eb8-4b19-acf4-91791e90fae6 None admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:30.787119 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e111189d-0eb8-4b19-acf4-91791e90fae6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:57:30.787472 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e111189d-0eb8-4b19-acf4-91791e90fae6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:57:30.788998 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e111189d-0eb8-4b19-acf4-91791e90fae6 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:57:30.792254 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd88825e-de90-4eb3-8175-c9285e07248a None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:57:30.813766 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 8/16] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 6 19:57:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:57:30.821975 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e111189d-0eb8-4b19-acf4-91791e90fae6 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:57:30.846539 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4048dcd6-b325-4c31-9b6b-cbd72f0b6082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:57:30.848023 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4048dcd6-b325-4c31-9b6b-cbd72f0b6082 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:30.848514 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4048dcd6-b325-4c31-9b6b-cbd72f0b6082 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:30.848972 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4048dcd6-b325-4c31-9b6b-cbd72f0b6082 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:30.888306 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4048dcd6-b325-4c31-9b6b-cbd72f0b6082 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:57:30.891140 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4048dcd6-b325-4c31-9b6b-cbd72f0b6082 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:57:30.928588 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-4048dcd6-b325-4c31-9b6b-cbd72f0b6082 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 19:57:30.928588 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:57:30.928588 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:57:30.928588 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:57:30.928588 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:57:30.928588 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:57:30.928588 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:57:30.928588 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:57:30.928588 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:57:30.928588 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:57:30.928588 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:57:30.928588 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:57:30.928588 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 19:57:30.928588 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 19:57:30.928588 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 19:57:30.928588 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 19:57:30.929593 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:57:30.929593 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:57:30.929593 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:57:30.929593 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:57:30.929593 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:57:30.929593 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:57:30.929593 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:57:30.929593 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 19:57:30.929593 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:57:30.929593 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:57:30.929593 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:57:30.929593 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:57:30.929593 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:57:30.929593 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:57:30.929593 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:57:30.929593 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:57:30.929593 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 19:57:30.930459 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 19:57:30.930459 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 19:57:30.930459 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 19:57:30.931346 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 9/17] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 19:57:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 19:57:30.942776 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4048dcd6-b325-4c31-9b6b-cbd72f0b6082 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:57:30.968099 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:57:31.017116 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e111189d-0eb8-4b19-acf4-91791e90fae6 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:57:31.070306 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0ff03a-2b27-4813-97b8-620eedda5122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:57:31.070974 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0ff03a-2b27-4813-97b8-620eedda5122 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:31.071324 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0ff03a-2b27-4813-97b8-620eedda5122 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:31.071686 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0ff03a-2b27-4813-97b8-620eedda5122 None admin] PATH_INFO: `/v3/users/1e7095a0137b45c984179dcf25de9362` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:31.073146 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0ff03a-2b27-4813-97b8-620eedda5122 None admin] RBAC: Authorizing `identity:get_user(user_id=1e7095a0137b45c984179dcf25de9362)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:57:31.074866 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0ff03a-2b27-4813-97b8-620eedda5122 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:57:31.076602 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 9/18] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 6 19:57:30 2022] GET /identity/v3/users/1e7095a0137b45c984179dcf25de9362 => generated 247 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 19:57:31.083399 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0ff03a-2b27-4813-97b8-620eedda5122 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:57:31.089119 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4048dcd6-b325-4c31-9b6b-cbd72f0b6082 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:57:31.124554 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:57:31.138584 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9543d295-0608-4d93-86b1-40dbba3077a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:57:31.139554 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9543d295-0608-4d93-86b1-40dbba3077a3 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:31.140066 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9543d295-0608-4d93-86b1-40dbba3077a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:31.140568 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9543d295-0608-4d93-86b1-40dbba3077a3 None admin] PATH_INFO: `/v3/domains` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:31.141862 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9543d295-0608-4d93-86b1-40dbba3077a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:57:31.143349 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9543d295-0608-4d93-86b1-40dbba3077a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:57:31.145951 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9543d295-0608-4d93-86b1-40dbba3077a3 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:57:31.180835 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 10/19] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 6 19:57:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 19:57:31.240949 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0ff03a-2b27-4813-97b8-620eedda5122 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:57:31.290769 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db90fbb0-c648-46d0-94c1-c5ad175dc337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:57:31.291483 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db90fbb0-c648-46d0-94c1-c5ad175dc337 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:31.291774 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db90fbb0-c648-46d0-94c1-c5ad175dc337 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:31.292310 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db90fbb0-c648-46d0-94c1-c5ad175dc337 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:31.293878 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db90fbb0-c648-46d0-94c1-c5ad175dc337 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:57:31.295658 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db90fbb0-c648-46d0-94c1-c5ad175dc337 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:57:31.297585 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 10/20] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 19:57:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 19:57:31.305168 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9543d295-0608-4d93-86b1-40dbba3077a3 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:57:31.492377 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9543d295-0608-4d93-86b1-40dbba3077a3 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:57:31.550260 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3ca283-f2ad-4f3e-a50a-ab2d0ae2cfef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:57:31.550910 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3ca283-f2ad-4f3e-a50a-ab2d0ae2cfef None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:31.551197 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3ca283-f2ad-4f3e-a50a-ab2d0ae2cfef None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:31.551197 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3ca283-f2ad-4f3e-a50a-ab2d0ae2cfef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:31.552408 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3ca283-f2ad-4f3e-a50a-ab2d0ae2cfef None admin] RBAC: Adding query filter params (role.id=e0ab41d8ae22474688b5361bcc75e0f0, scope.domain.id=default, user.id=1e7095a0137b45c984179dcf25de9362) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:57:31.552539 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3ca283-f2ad-4f3e-a50a-ab2d0ae2cfef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:57:31.554237 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3ca283-f2ad-4f3e-a50a-ab2d0ae2cfef None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:57:31.574612 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 11/21] 10.222.0.20 () {56 vars in 1322 bytes} [Tue Sep 6 19:57:31 2022] GET /identity/v3/role_assignments?user.id=1e7095a0137b45c984179dcf25de9362&scope.domain.id=default&role.id=e0ab41d8ae22474688b5361bcc75e0f0 => generated 233 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:57:34.506276 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1990cd-e9d4-4216-bff6-a7c1f6bd1190 None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:34.507269 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1990cd-e9d4-4216-bff6-a7c1f6bd1190 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:34.507761 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1990cd-e9d4-4216-bff6-a7c1f6bd1190 None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:34.509553 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 11/22] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 19:57:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 19:57:34.523056 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cc19eb-10bd-43d9-971c-7d730835a4aa None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:34.524031 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cc19eb-10bd-43d9-971c-7d730835a4aa None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:34.524550 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cc19eb-10bd-43d9-971c-7d730835a4aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:34.549624 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG passlib.handlers.bcrypt [None req-a5cc19eb-10bd-43d9-971c-7d730835a4aa None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81771) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 06 19:57:34.550395 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG passlib.handlers.bcrypt [None req-a5cc19eb-10bd-43d9-971c-7d730835a4aa None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81771) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 06 19:57:34.619595 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-a5cc19eb-10bd-43d9-971c-7d730835a4aa None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 19:57:34.621809 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-a5cc19eb-10bd-43d9-971c-7d730835a4aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 19:57:34.785382 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:57:34.839808 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 12/23] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 19:57:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 19:57:35.180633 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee31cee-98d8-466c-904a-a94bf333724a None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:35.181722 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee31cee-98d8-466c-904a-a94bf333724a None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:35.182513 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee31cee-98d8-466c-904a-a94bf333724a None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:35.185472 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 12/24] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 19:57:35 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 19:57:35.193987 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbb6db2-e67c-4246-9aa0-ce8ab063a227 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:35.194929 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbb6db2-e67c-4246-9aa0-ce8ab063a227 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:35.195679 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbb6db2-e67c-4246-9aa0-ce8ab063a227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:35.238469 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-fdbb6db2-e67c-4246-9aa0-ce8ab063a227 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 19:57:35.241116 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-fdbb6db2-e67c-4246-9aa0-ce8ab063a227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 19:57:35.300978 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43ffc1a-d185-44e1-b5ef-5384a399d45f None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:35.302040 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43ffc1a-d185-44e1-b5ef-5384a399d45f None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:35.302847 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43ffc1a-d185-44e1-b5ef-5384a399d45f None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:35.306356 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 13/25] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 19:57:35 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 19:57:35.326331 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b60b1c-edfb-4b7b-84ef-d602fe5ad1ee None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:35.327253 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b60b1c-edfb-4b7b-84ef-d602fe5ad1ee None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:35.328008 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b60b1c-edfb-4b7b-84ef-d602fe5ad1ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:35.366729 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 13/26] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 19:57:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 19:57:35.367980 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbb6db2-e67c-4246-9aa0-ce8ab063a227 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:57:35.383251 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-17b60b1c-edfb-4b7b-84ef-d602fe5ad1ee None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 19:57:35.394587 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-17b60b1c-edfb-4b7b-84ef-d602fe5ad1ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 19:57:35.400940 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:57:35.557917 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbb6db2-e67c-4246-9aa0-ce8ab063a227 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:57:35.578405 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:57:35.589421 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 14/27] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 19:57:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 19:57:35.621877 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc912ef-137a-41df-a9be-46bdb5768084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:57:35.622641 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc912ef-137a-41df-a9be-46bdb5768084 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:35.622936 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc912ef-137a-41df-a9be-46bdb5768084 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:35.623223 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc912ef-137a-41df-a9be-46bdb5768084 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:35.656557 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc912ef-137a-41df-a9be-46bdb5768084 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:57:35.658210 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc912ef-137a-41df-a9be-46bdb5768084 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:57:35.692136 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-2dc912ef-137a-41df-a9be-46bdb5768084 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 19:57:35.692136 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:57:35.692136 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:57:35.692136 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:57:35.692136 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:57:35.692136 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:57:35.692136 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:57:35.692136 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:57:35.692136 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:57:35.692136 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:57:35.692136 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:57:35.692136 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:57:35.692136 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 19:57:35.692136 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 19:57:35.692136 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 19:57:35.692136 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 19:57:35.693841 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:57:35.693841 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:57:35.693841 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:57:35.693841 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:57:35.693841 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:57:35.693841 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:57:35.693841 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:57:35.693841 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 19:57:35.693841 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:57:35.693841 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:57:35.693841 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:57:35.693841 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:57:35.693841 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:57:35.693841 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:57:35.693841 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:57:35.693841 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:57:35.693841 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 19:57:35.695376 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 19:57:35.695376 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 19:57:35.695376 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 19:57:35.695376 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 14/28] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 19:57:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:57:35.700143 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b60b1c-edfb-4b7b-84ef-d602fe5ad1ee None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:57:35.838106 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc912ef-137a-41df-a9be-46bdb5768084 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:57:35.920344 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b60b1c-edfb-4b7b-84ef-d602fe5ad1ee None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:57:36.026635 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891df251-1f2f-4468-b80e-7d51c3ddcb4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:57:36.027344 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891df251-1f2f-4468-b80e-7d51c3ddcb4d None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:36.027634 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891df251-1f2f-4468-b80e-7d51c3ddcb4d None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:36.027974 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891df251-1f2f-4468-b80e-7d51c3ddcb4d None admin] PATH_INFO: `/v3/domains` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:36.028553 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891df251-1f2f-4468-b80e-7d51c3ddcb4d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:57:36.028869 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891df251-1f2f-4468-b80e-7d51c3ddcb4d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:57:36.030383 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891df251-1f2f-4468-b80e-7d51c3ddcb4d None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:57:36.066531 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:57:36.075948 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 15/29] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 6 19:57:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 19:57:36.086516 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891df251-1f2f-4468-b80e-7d51c3ddcb4d None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:57:36.134502 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc912ef-137a-41df-a9be-46bdb5768084 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:57:36.245917 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b69b98-d3d6-4452-9e9c-aa97a99a1bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:57:36.247792 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b69b98-d3d6-4452-9e9c-aa97a99a1bcb None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:36.248518 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b69b98-d3d6-4452-9e9c-aa97a99a1bcb None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:36.249021 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b69b98-d3d6-4452-9e9c-aa97a99a1bcb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:36.283391 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b69b98-d3d6-4452-9e9c-aa97a99a1bcb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 19:57:36.312603 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-73b69b98-d3d6-4452-9e9c-aa97a99a1bcb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 19:57:36.312603 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:57:36.312603 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:57:36.312603 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:57:36.312603 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:57:36.312603 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:57:36.312603 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:57:36.312603 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:57:36.312603 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:57:36.312603 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:57:36.312603 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:57:36.312603 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:57:36.312603 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 19:57:36.312603 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 19:57:36.312603 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 19:57:36.312603 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 19:57:36.313703 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 19:57:36.313703 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 19:57:36.313703 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 19:57:36.313703 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 19:57:36.313703 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:57:36.313703 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:57:36.313703 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:57:36.313703 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:57:36.313703 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:57:36.313703 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:57:36.313703 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:57:36.313703 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 19:57:36.313703 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:57:36.313703 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:57:36.313703 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:57:36.313703 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:57:36.314572 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:57:36.314572 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:57:36.314572 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:57:36.314572 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:57:36.314572 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 19:57:36.314572 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 19:57:36.314572 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 19:57:36.314572 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 19:57:36.314572 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 19:57:36.314572 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 19:57:36.314572 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 19:57:36.314572 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 19:57:36.316041 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 15/30] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Sep 6 19:57:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 478 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 19:57:36.317232 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b69b98-d3d6-4452-9e9c-aa97a99a1bcb None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:57:36.368119 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891df251-1f2f-4468-b80e-7d51c3ddcb4d None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:57:36.391961 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:57:36.463252 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ae39c1-bb7c-40a9-8a05-e15ce48e216d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:57:36.463903 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ae39c1-bb7c-40a9-8a05-e15ce48e216d None admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:36.464190 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ae39c1-bb7c-40a9-8a05-e15ce48e216d None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:36.464482 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ae39c1-bb7c-40a9-8a05-e15ce48e216d None admin] PATH_INFO: `/v3/projects` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:36.465350 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ae39c1-bb7c-40a9-8a05-e15ce48e216d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:57:36.467020 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ae39c1-bb7c-40a9-8a05-e15ce48e216d None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:57:36.640332 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b69b98-d3d6-4452-9e9c-aa97a99a1bcb None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:57:36.688786 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 16/31] 10.222.0.20 () {60 vars in 1141 bytes} [Tue Sep 6 19:57:36 2022] POST /identity/v3/projects => generated 301 bytes in 603 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 19:57:36.691029 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ae39c1-bb7c-40a9-8a05-e15ce48e216d None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:57:36.731793 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5797a7be-6eb7-45b8-9deb-a37e6915429e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:57:36.735650 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5797a7be-6eb7-45b8-9deb-a37e6915429e None admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:36.736056 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5797a7be-6eb7-45b8-9deb-a37e6915429e None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:36.736331 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5797a7be-6eb7-45b8-9deb-a37e6915429e None admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:36.737053 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5797a7be-6eb7-45b8-9deb-a37e6915429e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:57:36.738744 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5797a7be-6eb7-45b8-9deb-a37e6915429e None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:57:36.812361 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:57:36.888700 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ae39c1-bb7c-40a9-8a05-e15ce48e216d None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:57:36.937268 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b97a827-0441-4bc1-8e6d-4d0eee5fc243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:57:36.939009 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b97a827-0441-4bc1-8e6d-4d0eee5fc243 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:36.939688 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b97a827-0441-4bc1-8e6d-4d0eee5fc243 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:36.940319 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b97a827-0441-4bc1-8e6d-4d0eee5fc243 None admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:36.941693 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b97a827-0441-4bc1-8e6d-4d0eee5fc243 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:57:36.942239 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b97a827-0441-4bc1-8e6d-4d0eee5fc243 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:57:36.944097 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b97a827-0441-4bc1-8e6d-4d0eee5fc243 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:57:36.957131 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 17/32] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 6 19:57:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:57:36.959283 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c93595-f979-4ecc-8ce1-e93252c01e7a None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:36.959781 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c93595-f979-4ecc-8ce1-e93252c01e7a None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:36.960327 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c93595-f979-4ecc-8ce1-e93252c01e7a None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:36.962019 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 18/33] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 19:57:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 19:57:36.964073 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf0fd4b-0a3d-4f01-b208-4f4d05a07c0e None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:36.964618 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf0fd4b-0a3d-4f01-b208-4f4d05a07c0e None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:36.965057 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf0fd4b-0a3d-4f01-b208-4f4d05a07c0e None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:36.967327 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 19/34] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 19:57:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 19:57:36.968683 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf0fd4b-0a3d-4f01-b208-4f4d05a07c0e None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:57:36.975517 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 16/35] 10.222.0.20 () {60 vars in 1131 bytes} [Tue Sep 6 19:57:36 2022] POST /identity/v3/roles => generated 219 bytes in 660 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 19:57:36.977434 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c367b7a-7a10-4f29-8d51-50ef828863fb None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:36.977977 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c367b7a-7a10-4f29-8d51-50ef828863fb None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:36.978377 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c367b7a-7a10-4f29-8d51-50ef828863fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:37.029892 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-3c367b7a-7a10-4f29-8d51-50ef828863fb None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 19:57:37.033900 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-3c367b7a-7a10-4f29-8d51-50ef828863fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 19:57:37.116598 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf0fd4b-0a3d-4f01-b208-4f4d05a07c0e None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:57:37.142080 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:57:37.168298 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c446a42-6841-41ea-a8ac-289ee3ec6f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:57:37.168981 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c446a42-6841-41ea-a8ac-289ee3ec6f7a None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:37.169327 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c446a42-6841-41ea-a8ac-289ee3ec6f7a None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:37.169643 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c446a42-6841-41ea-a8ac-289ee3ec6f7a None admin] PATH_INFO: `/v3/users/1e7095a0137b45c984179dcf25de9362` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:37.171699 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c446a42-6841-41ea-a8ac-289ee3ec6f7a None admin] RBAC: Authorizing `identity:get_user(user_id=1e7095a0137b45c984179dcf25de9362)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:57:37.173378 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c446a42-6841-41ea-a8ac-289ee3ec6f7a None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:57:37.175509 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 20/36] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 6 19:57:36 2022] GET /identity/v3/users/1e7095a0137b45c984179dcf25de9362 => generated 247 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:57:37.178539 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8a6bb6-f754-4df0-a36e-bed5cc44d8ce None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:37.179209 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8a6bb6-f754-4df0-a36e-bed5cc44d8ce None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:37.179738 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8a6bb6-f754-4df0-a36e-bed5cc44d8ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:37.199728 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 17/37] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 19:57:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 19:57:37.201723 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c367b7a-7a10-4f29-8d51-50ef828863fb None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:57:37.204391 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-7e8a6bb6-f754-4df0-a36e-bed5cc44d8ce None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 19:57:37.206527 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-7e8a6bb6-f754-4df0-a36e-bed5cc44d8ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 19:57:37.230843 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:57:37.308695 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 21/38] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 19:57:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 19:57:37.412359 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c367b7a-7a10-4f29-8d51-50ef828863fb None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:57:37.472398 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8311049-c298-4ee9-b6fe-0adc91dee1cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:57:37.474006 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8311049-c298-4ee9-b6fe-0adc91dee1cc None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:37.474782 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8311049-c298-4ee9-b6fe-0adc91dee1cc None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:37.475541 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8311049-c298-4ee9-b6fe-0adc91dee1cc None admin] PATH_INFO: `/v3/domains/default` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:37.478566 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8311049-c298-4ee9-b6fe-0adc91dee1cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:57:37.481946 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8311049-c298-4ee9-b6fe-0adc91dee1cc None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:57:37.485597 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 18/39] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 19:57:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:57:37.493024 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8a6bb6-f754-4df0-a36e-bed5cc44d8ce None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:57:37.501412 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c857df1e-a8bd-490d-96cd-818e3bf8afab None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:37.501412 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c857df1e-a8bd-490d-96cd-818e3bf8afab None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:37.501412 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c857df1e-a8bd-490d-96cd-818e3bf8afab None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:37.502311 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 19/40] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 19:57:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 19:57:37.509748 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbb0526-4193-40f8-a167-704f5b6e056f None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:37.510264 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbb0526-4193-40f8-a167-704f5b6e056f None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:37.510780 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbb0526-4193-40f8-a167-704f5b6e056f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:37.548359 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-ddbb0526-4193-40f8-a167-704f5b6e056f None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 19:57:37.550032 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-ddbb0526-4193-40f8-a167-704f5b6e056f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 19:57:37.694672 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 20/41] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 19:57:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 19:57:37.695646 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbb0526-4193-40f8-a167-704f5b6e056f None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:57:37.719900 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:57:37.734595 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8a6bb6-f754-4df0-a36e-bed5cc44d8ce None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:57:37.765697 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:57:37.799263 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89634c2f-87de-4931-b96d-19b229ead4f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:57:37.800281 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89634c2f-87de-4931-b96d-19b229ead4f5 None admin] REQUEST_METHOD: `PUT` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:37.800757 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89634c2f-87de-4931-b96d-19b229ead4f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:37.801092 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89634c2f-87de-4931-b96d-19b229ead4f5 None admin] PATH_INFO: `/v3/domains/default/users/1e7095a0137b45c984179dcf25de9362/roles/e0ab41d8ae22474688b5361bcc75e0f0` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:37.803593 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89634c2f-87de-4931-b96d-19b229ead4f5 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=1e7095a0137b45c984179dcf25de9362, role_id=e0ab41d8ae22474688b5361bcc75e0f0)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:57:37.805168 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89634c2f-87de-4931-b96d-19b229ead4f5 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:57:37.830886 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 22/42] 10.222.0.20 () {58 vars in 1361 bytes} [Tue Sep 6 19:57:37 2022] PUT /identity/v3/domains/default/users/1e7095a0137b45c984179dcf25de9362/roles/e0ab41d8ae22474688b5361bcc75e0f0 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 19:57:37.830886 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89634c2f-87de-4931-b96d-19b229ead4f5 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:57:37.893556 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbb0526-4193-40f8-a167-704f5b6e056f None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:57:37.971670 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89634c2f-87de-4931-b96d-19b229ead4f5 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:57:37.977211 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4993d615-d671-4797-8982-c38d1bf26d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:57:37.978506 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4993d615-d671-4797-8982-c38d1bf26d2d None admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:37.979130 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4993d615-d671-4797-8982-c38d1bf26d2d None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:37.979768 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4993d615-d671-4797-8982-c38d1bf26d2d None admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:37.981069 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4993d615-d671-4797-8982-c38d1bf26d2d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:57:37.984799 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4993d615-d671-4797-8982-c38d1bf26d2d None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:57:38.006921 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 21/43] 10.222.0.20 () {60 vars in 1131 bytes} [Tue Sep 6 19:57:37 2022] POST /identity/v3/roles => generated 223 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 19:57:38.016421 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4993d615-d671-4797-8982-c38d1bf26d2d None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:57:38.027912 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b30c64-83af-46c5-9dae-34bd338f4502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:57:38.029467 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b30c64-83af-46c5-9dae-34bd338f4502 None admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:38.029925 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b30c64-83af-46c5-9dae-34bd338f4502 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:38.030309 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b30c64-83af-46c5-9dae-34bd338f4502 None admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:38.031181 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b30c64-83af-46c5-9dae-34bd338f4502 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:57:38.033409 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b30c64-83af-46c5-9dae-34bd338f4502 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:57:38.056052 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 23/44] 10.222.0.20 () {60 vars in 1131 bytes} [Tue Sep 6 19:57:37 2022] POST /identity/v3/roles => generated 225 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 19:57:38.176084 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:57:38.240330 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4993d615-d671-4797-8982-c38d1bf26d2d None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:57:38.303103 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc06b92-3c99-4343-a135-bcf7d9c57796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:57:38.304403 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc06b92-3c99-4343-a135-bcf7d9c57796 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:38.304931 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc06b92-3c99-4343-a135-bcf7d9c57796 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:38.305431 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc06b92-3c99-4343-a135-bcf7d9c57796 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:38.307666 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc06b92-3c99-4343-a135-bcf7d9c57796 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:57:38.310852 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc06b92-3c99-4343-a135-bcf7d9c57796 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:57:38.313809 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 22/45] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 19:57:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:57:38.323456 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b30c64-83af-46c5-9dae-34bd338f4502 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:57:38.379012 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:57:38.466018 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b30c64-83af-46c5-9dae-34bd338f4502 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:57:38.505593 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a79453c-494c-4592-a5ba-154a2901828c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:57:38.506299 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a79453c-494c-4592-a5ba-154a2901828c None admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:38.506650 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a79453c-494c-4592-a5ba-154a2901828c None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:38.507010 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a79453c-494c-4592-a5ba-154a2901828c None admin] PATH_INFO: `/v3/projects` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:38.507758 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a79453c-494c-4592-a5ba-154a2901828c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:57:38.509328 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a79453c-494c-4592-a5ba-154a2901828c None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:57:38.541099 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 24/46] 10.222.0.20 () {60 vars in 1141 bytes} [Tue Sep 6 19:57:38 2022] POST /identity/v3/projects => generated 312 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 19:57:40.358668 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0387f9bf-0555-46a0-839b-1632e4314c26 None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:40.359475 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0387f9bf-0555-46a0-839b-1632e4314c26 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:40.359475 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0387f9bf-0555-46a0-839b-1632e4314c26 None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:40.360864 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 23/47] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 19:57:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 19:57:40.367216 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384684d1-412c-4485-90e7-4e8b7210ff9d None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:40.367979 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384684d1-412c-4485-90e7-4e8b7210ff9d None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:40.367979 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384684d1-412c-4485-90e7-4e8b7210ff9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:40.391151 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-384684d1-412c-4485-90e7-4e8b7210ff9d None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 19:57:40.392652 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-384684d1-412c-4485-90e7-4e8b7210ff9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 19:57:40.599644 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 25/48] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 19:57:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 19:57:40.896148 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0387f9bf-0555-46a0-839b-1632e4314c26 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:57:41.047806 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0387f9bf-0555-46a0-839b-1632e4314c26 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:57:41.093634 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d11be50-d679-4dd3-88ca-9fe393270278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:57:41.094376 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d11be50-d679-4dd3-88ca-9fe393270278 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:41.094751 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d11be50-d679-4dd3-88ca-9fe393270278 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:41.094928 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d11be50-d679-4dd3-88ca-9fe393270278 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:41.104251 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:57:41.119150 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d11be50-d679-4dd3-88ca-9fe393270278 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 19:57:41.124592 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-3d11be50-d679-4dd3-88ca-9fe393270278 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 19:57:41.124592 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:57:41.124592 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:57:41.124592 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:57:41.124592 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:57:41.124592 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:57:41.124592 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:57:41.124592 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:57:41.124592 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:57:41.124592 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:57:41.124592 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:57:41.124592 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:57:41.124592 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 19:57:41.124592 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 19:57:41.124592 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 19:57:41.124592 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 19:57:41.126067 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 19:57:41.126067 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 19:57:41.126067 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 19:57:41.126067 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 19:57:41.126067 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:57:41.126067 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:57:41.126067 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:57:41.126067 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:57:41.126067 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:57:41.126067 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:57:41.126067 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:57:41.126067 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 19:57:41.126067 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:57:41.126067 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:57:41.126067 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:57:41.126067 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:57:41.127294 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:57:41.127294 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:57:41.127294 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:57:41.127294 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:57:41.127294 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 19:57:41.127294 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 19:57:41.127294 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 19:57:41.127294 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 19:57:41.127294 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 19:57:41.127294 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 19:57:41.127294 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 19:57:41.127294 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 19:57:41.127294 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 24/49] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Sep 6 19:57:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:57:41.133022 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384684d1-412c-4485-90e7-4e8b7210ff9d None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:57:41.155876 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:57:41.221143 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb0efef-82e5-4bd1-a455-68e7f046867c None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:41.221460 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb0efef-82e5-4bd1-a455-68e7f046867c None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:41.221767 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb0efef-82e5-4bd1-a455-68e7f046867c None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:41.223163 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 25/50] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 19:57:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 19:57:41.231993 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec38a41e-d6b9-4c87-8e9b-919011e0f532 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:41.232434 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec38a41e-d6b9-4c87-8e9b-919011e0f532 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:41.232984 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec38a41e-d6b9-4c87-8e9b-919011e0f532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:41.253564 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-ec38a41e-d6b9-4c87-8e9b-919011e0f532 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 19:57:41.254829 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-ec38a41e-d6b9-4c87-8e9b-919011e0f532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 19:57:41.297250 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384684d1-412c-4485-90e7-4e8b7210ff9d None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:57:41.341202 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 26/51] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 19:57:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 19:57:41.344390 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb79e5a-eadb-4da7-990d-88af0cd57238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:57:41.345032 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb79e5a-eadb-4da7-990d-88af0cd57238 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:41.345324 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb79e5a-eadb-4da7-990d-88af0cd57238 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:41.345627 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb79e5a-eadb-4da7-990d-88af0cd57238 None admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:41.346329 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb79e5a-eadb-4da7-990d-88af0cd57238 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:57:41.346647 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb79e5a-eadb-4da7-990d-88af0cd57238 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:57:41.348095 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb79e5a-eadb-4da7-990d-88af0cd57238 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:57:41.358510 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 26/52] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 6 19:57:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:57:41.364493 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec38a41e-d6b9-4c87-8e9b-919011e0f532 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:57:41.498465 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec38a41e-d6b9-4c87-8e9b-919011e0f532 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:57:41.647210 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:57:41.658173 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb79e5a-eadb-4da7-990d-88af0cd57238 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:57:41.662161 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4783dff-8d39-4c41-ade4-534b59bf90a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:57:41.663148 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4783dff-8d39-4c41-ade4-534b59bf90a7 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:41.663610 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4783dff-8d39-4c41-ade4-534b59bf90a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:41.664074 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4783dff-8d39-4c41-ade4-534b59bf90a7 None admin] PATH_INFO: `/v3/users/1e7095a0137b45c984179dcf25de9362` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:41.666490 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4783dff-8d39-4c41-ade4-534b59bf90a7 None admin] RBAC: Authorizing `identity:get_user(user_id=1e7095a0137b45c984179dcf25de9362)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:57:41.668539 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4783dff-8d39-4c41-ade4-534b59bf90a7 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:57:41.670849 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 27/53] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 6 19:57:41 2022] GET /identity/v3/users/1e7095a0137b45c984179dcf25de9362 => generated 247 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:57:41.679366 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4783dff-8d39-4c41-ade4-534b59bf90a7 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:57:41.773729 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:57:41.803785 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb79e5a-eadb-4da7-990d-88af0cd57238 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:57:41.841644 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4783dff-8d39-4c41-ade4-534b59bf90a7 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:57:41.845883 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eefb9b0-3604-4e1d-81e2-dcf9f3be8d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:57:41.846553 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eefb9b0-3604-4e1d-81e2-dcf9f3be8d2b None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:41.846835 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eefb9b0-3604-4e1d-81e2-dcf9f3be8d2b None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:41.847112 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eefb9b0-3604-4e1d-81e2-dcf9f3be8d2b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:41.848421 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eefb9b0-3604-4e1d-81e2-dcf9f3be8d2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:57:41.850118 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eefb9b0-3604-4e1d-81e2-dcf9f3be8d2b None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:57:41.851711 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 27/54] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 19:57:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:57:41.860736 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eefb9b0-3604-4e1d-81e2-dcf9f3be8d2b None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:57:41.897337 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ab68db-3979-472e-aacb-c3bd891e04c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:57:41.898114 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ab68db-3979-472e-aacb-c3bd891e04c1 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:41.898397 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ab68db-3979-472e-aacb-c3bd891e04c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:41.898671 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ab68db-3979-472e-aacb-c3bd891e04c1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:41.900584 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ab68db-3979-472e-aacb-c3bd891e04c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:57:41.902252 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ab68db-3979-472e-aacb-c3bd891e04c1 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:57:41.904036 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 28/55] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 19:57:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 19:57:41.910648 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ab68db-3979-472e-aacb-c3bd891e04c1 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:57:41.998446 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eefb9b0-3604-4e1d-81e2-dcf9f3be8d2b None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:57:42.040411 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:57:42.047071 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd0f957-1d70-458c-83bd-50c14d5dae21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:57:42.048059 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd0f957-1d70-458c-83bd-50c14d5dae21 None admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:42.048519 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd0f957-1d70-458c-83bd-50c14d5dae21 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:42.048908 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd0f957-1d70-458c-83bd-50c14d5dae21 None admin] PATH_INFO: `/v3/projects` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:42.050109 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd0f957-1d70-458c-83bd-50c14d5dae21 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:57:42.052459 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd0f957-1d70-458c-83bd-50c14d5dae21 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:57:42.077716 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:57:42.094297 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 28/56] 10.222.0.20 () {60 vars in 1140 bytes} [Tue Sep 6 19:57:41 2022] POST /identity/v3/projects => generated 298 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 19:57:42.104237 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ab68db-3979-472e-aacb-c3bd891e04c1 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:57:42.153024 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125e0949-85d2-47ff-8031-6ed753322a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:57:42.153493 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125e0949-85d2-47ff-8031-6ed753322a11 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:42.153809 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125e0949-85d2-47ff-8031-6ed753322a11 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:42.154093 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125e0949-85d2-47ff-8031-6ed753322a11 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:42.154862 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125e0949-85d2-47ff-8031-6ed753322a11 None admin] RBAC: Adding query filter params (role.id=e0ab41d8ae22474688b5361bcc75e0f0, scope.domain.id=default, user.id=1e7095a0137b45c984179dcf25de9362) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:57:42.155173 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125e0949-85d2-47ff-8031-6ed753322a11 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:57:42.156734 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125e0949-85d2-47ff-8031-6ed753322a11 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:57:42.165240 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 29/57] 10.222.0.20 () {56 vars in 1322 bytes} [Tue Sep 6 19:57:41 2022] GET /identity/v3/role_assignments?user.id=1e7095a0137b45c984179dcf25de9362&scope.domain.id=default&role.id=e0ab41d8ae22474688b5361bcc75e0f0 => generated 530 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 19:57:44.695333 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5ad8d6-eb1b-4c30-905d-e7833c79e7fd None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:44.695865 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5ad8d6-eb1b-4c30-905d-e7833c79e7fd None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:44.696056 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5ad8d6-eb1b-4c30-905d-e7833c79e7fd None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:44.697566 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 29/58] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 19:57:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 19:57:44.706934 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a81522-9ab4-4e84-8def-260641645641 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:44.707532 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a81522-9ab4-4e84-8def-260641645641 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:44.707836 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a81522-9ab4-4e84-8def-260641645641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:44.738867 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-a4a81522-9ab4-4e84-8def-260641645641 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 19:57:44.741578 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-a4a81522-9ab4-4e84-8def-260641645641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 19:57:44.892038 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 30/59] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 19:57:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 19:57:45.208607 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5ad8d6-eb1b-4c30-905d-e7833c79e7fd None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:57:45.378362 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5ad8d6-eb1b-4c30-905d-e7833c79e7fd None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:57:45.422228 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fc4635-22fc-4c81-8ca1-22f96c70c668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:57:45.422960 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fc4635-22fc-4c81-8ca1-22f96c70c668 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:45.423295 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fc4635-22fc-4c81-8ca1-22f96c70c668 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:45.423624 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fc4635-22fc-4c81-8ca1-22f96c70c668 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:45.424958 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fc4635-22fc-4c81-8ca1-22f96c70c668 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:57:45.426714 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fc4635-22fc-4c81-8ca1-22f96c70c668 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:57:45.428569 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 30/60] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 19:57:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 19:57:45.434788 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a81522-9ab4-4e84-8def-260641645641 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:57:45.509128 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:57:45.559672 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a81522-9ab4-4e84-8def-260641645641 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:57:45.600195 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ba13d3-56f0-42b6-b294-fc973a8e41fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:57:45.600876 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ba13d3-56f0-42b6-b294-fc973a8e41fb None admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:45.601287 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ba13d3-56f0-42b6-b294-fc973a8e41fb None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:45.601647 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ba13d3-56f0-42b6-b294-fc973a8e41fb None admin] PATH_INFO: `/v3/users` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:45.602423 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ba13d3-56f0-42b6-b294-fc973a8e41fb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:57:45.604026 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ba13d3-56f0-42b6-b294-fc973a8e41fb None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:57:45.638437 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 31/61] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Sep 6 19:57:45 2022] POST /identity/v3/users => generated 275 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 19:57:50.428295 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeef248b-d2db-4090-85e3-a8dc56af2b04 None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:50.429457 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeef248b-d2db-4090-85e3-a8dc56af2b04 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:50.430014 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeef248b-d2db-4090-85e3-a8dc56af2b04 None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:50.432094 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 31/62] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 19:57:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 19:57:50.434052 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb21d19-6d92-4d3a-8f5b-c20e19d4d6d5 None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:50.435012 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb21d19-6d92-4d3a-8f5b-c20e19d4d6d5 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:50.435509 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb21d19-6d92-4d3a-8f5b-c20e19d4d6d5 None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:50.437220 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 32/63] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 19:57:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 19:57:50.440623 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dea36d-fb3b-458c-b266-40951fdada59 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:50.441245 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dea36d-fb3b-458c-b266-40951fdada59 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:50.441777 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dea36d-fb3b-458c-b266-40951fdada59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:50.451542 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd1a22f-26d4-472f-a1e1-77271986e4b4 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:50.453770 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd1a22f-26d4-472f-a1e1-77271986e4b4 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:50.454342 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd1a22f-26d4-472f-a1e1-77271986e4b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:50.488313 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-25dea36d-fb3b-458c-b266-40951fdada59 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 19:57:50.491072 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-25dea36d-fb3b-458c-b266-40951fdada59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 19:57:50.511555 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-9fd1a22f-26d4-472f-a1e1-77271986e4b4 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 19:57:50.515018 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-9fd1a22f-26d4-472f-a1e1-77271986e4b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 19:57:50.708868 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:57:50.720496 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 33/64] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 19:57:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 19:57:50.739251 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 32/65] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 19:57:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 19:57:50.749467 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f803e1-db9a-42de-979c-efaff23da657 None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:50.750526 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f803e1-db9a-42de-979c-efaff23da657 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:50.751314 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f803e1-db9a-42de-979c-efaff23da657 None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:50.758594 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 34/66] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 19:57:50 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 19:57:50.765265 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab4de6d-aacd-45e4-b7ef-b25ee9d7575b None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:50.766180 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab4de6d-aacd-45e4-b7ef-b25ee9d7575b None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:50.766418 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab4de6d-aacd-45e4-b7ef-b25ee9d7575b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:50.808862 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-bab4de6d-aacd-45e4-b7ef-b25ee9d7575b None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 19:57:50.811198 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-bab4de6d-aacd-45e4-b7ef-b25ee9d7575b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 19:57:50.928663 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 33/67] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 19:57:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 19:57:51.244311 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f803e1-db9a-42de-979c-efaff23da657 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:57:51.251433 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab4de6d-aacd-45e4-b7ef-b25ee9d7575b None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:57:51.318124 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:57:51.423324 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f803e1-db9a-42de-979c-efaff23da657 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:57:51.439469 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab4de6d-aacd-45e4-b7ef-b25ee9d7575b None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:57:51.468535 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:57:51.484588 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d59597a-9b80-48ce-92cc-6857c14aa799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:57:51.485250 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d59597a-9b80-48ce-92cc-6857c14aa799 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:51.485533 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d59597a-9b80-48ce-92cc-6857c14aa799 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:51.485895 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d59597a-9b80-48ce-92cc-6857c14aa799 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:51.489728 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0869fa1a-5675-48a4-80b4-5beb2e35c8ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:57:51.490702 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0869fa1a-5675-48a4-80b4-5beb2e35c8ab None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:51.491144 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0869fa1a-5675-48a4-80b4-5beb2e35c8ab None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:51.491653 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0869fa1a-5675-48a4-80b4-5beb2e35c8ab None admin] PATH_INFO: `/v3/roles/member` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:51.498630 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d59597a-9b80-48ce-92cc-6857c14aa799 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 19:57:51.503660 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-1d59597a-9b80-48ce-92cc-6857c14aa799 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 19:57:51.503660 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:57:51.503660 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:57:51.503660 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:57:51.503660 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:57:51.503660 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:57:51.503660 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:57:51.503660 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:57:51.503660 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:57:51.503660 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:57:51.503660 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:57:51.503660 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:57:51.503660 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 19:57:51.503660 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 19:57:51.503660 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 19:57:51.503660 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 19:57:51.504899 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 19:57:51.504899 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 19:57:51.504899 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 19:57:51.504899 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 19:57:51.504899 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:57:51.504899 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:57:51.504899 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:57:51.504899 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:57:51.504899 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:57:51.504899 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:57:51.504899 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:57:51.504899 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 19:57:51.504899 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:57:51.504899 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:57:51.504899 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:57:51.504899 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:57:51.505766 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:57:51.505766 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:57:51.505766 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:57:51.505766 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:57:51.505766 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 19:57:51.505766 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 19:57:51.505766 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 19:57:51.505766 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 19:57:51.505766 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 19:57:51.505766 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 19:57:51.505766 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 19:57:51.505766 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 19:57:51.505766 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 34/68] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Sep 6 19:57:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:57:51.507256 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef920019-fea6-4aef-8b64-b7b3bd2a6a63 None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:51.507256 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef920019-fea6-4aef-8b64-b7b3bd2a6a63 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:51.507256 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef920019-fea6-4aef-8b64-b7b3bd2a6a63 None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:51.508777 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 35/69] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 19:57:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 19:57:51.510971 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a176b6fb-525d-4ce9-85d9-400aee3e4d84 None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:51.511553 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a176b6fb-525d-4ce9-85d9-400aee3e4d84 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:51.512625 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a176b6fb-525d-4ce9-85d9-400aee3e4d84 None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:51.514570 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 36/70] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 19:57:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 19:57:51.515935 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a176b6fb-525d-4ce9-85d9-400aee3e4d84 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:57:51.517117 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0869fa1a-5675-48a4-80b4-5beb2e35c8ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 19:57:51.522999 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-0869fa1a-5675-48a4-80b4-5beb2e35c8ab None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 19:57:51.522999 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:57:51.522999 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:57:51.522999 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:57:51.522999 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:57:51.522999 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:57:51.522999 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:57:51.522999 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:57:51.522999 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:57:51.522999 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:57:51.522999 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:57:51.522999 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:57:51.522999 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 19:57:51.522999 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 19:57:51.522999 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 19:57:51.522999 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 19:57:51.525017 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 19:57:51.525017 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 19:57:51.525017 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 19:57:51.525017 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 19:57:51.525017 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:57:51.525017 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:57:51.525017 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:57:51.525017 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:57:51.525017 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:57:51.525017 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:57:51.525017 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:57:51.525017 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 19:57:51.525017 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:57:51.525017 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:57:51.525017 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:57:51.525017 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:57:51.526668 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:57:51.526668 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:57:51.526668 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:57:51.526668 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:57:51.526668 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 19:57:51.526668 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 19:57:51.526668 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 19:57:51.526668 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 19:57:51.526668 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 19:57:51.526668 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 19:57:51.526668 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 19:57:51.526668 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 19:57:51.526668 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 35/71] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 6 19:57:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 19:57:51.526668 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0869fa1a-5675-48a4-80b4-5beb2e35c8ab None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:57:51.650157 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0869fa1a-5675-48a4-80b4-5beb2e35c8ab None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:57:51.651926 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a176b6fb-525d-4ce9-85d9-400aee3e4d84 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:57:51.652843 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:57:51.695618 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef77597-321a-453b-be1c-56106d9ff8f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:57:51.696486 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef77597-321a-453b-be1c-56106d9ff8f9 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:51.696861 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef77597-321a-453b-be1c-56106d9ff8f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:51.697253 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef77597-321a-453b-be1c-56106d9ff8f9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:51.699544 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4e62db-18d0-42e8-9dd3-a34cbcc4a360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:57:51.700179 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4e62db-18d0-42e8-9dd3-a34cbcc4a360 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:51.700461 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4e62db-18d0-42e8-9dd3-a34cbcc4a360 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:51.700749 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4e62db-18d0-42e8-9dd3-a34cbcc4a360 None admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:51.701490 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4e62db-18d0-42e8-9dd3-a34cbcc4a360 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:57:51.701846 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4e62db-18d0-42e8-9dd3-a34cbcc4a360 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:57:51.703935 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4e62db-18d0-42e8-9dd3-a34cbcc4a360 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:57:51.712403 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef77597-321a-453b-be1c-56106d9ff8f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 19:57:51.717542 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-1ef77597-321a-453b-be1c-56106d9ff8f9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 19:57:51.717542 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:57:51.717542 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:57:51.717542 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:57:51.717542 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:57:51.717542 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:57:51.717542 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:57:51.717542 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:57:51.717542 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:57:51.717542 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:57:51.717542 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:57:51.717542 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:57:51.717542 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 19:57:51.717542 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 19:57:51.717542 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 19:57:51.717542 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 19:57:51.719632 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 19:57:51.719632 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 19:57:51.719632 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 19:57:51.719632 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 19:57:51.719632 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:57:51.719632 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:57:51.719632 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:57:51.719632 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:57:51.719632 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:57:51.719632 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:57:51.719632 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:57:51.719632 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 19:57:51.719632 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:57:51.719632 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:57:51.719632 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:57:51.719632 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:57:51.721112 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:57:51.721112 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:57:51.721112 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:57:51.721112 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:57:51.721112 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 19:57:51.721112 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 19:57:51.721112 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 19:57:51.721112 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 19:57:51.721112 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 19:57:51.721112 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 19:57:51.721112 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 19:57:51.721112 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 19:57:51.722288 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 36/72] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 6 19:57:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:57:51.722418 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 37/73] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Sep 6 19:57:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:57:51.722543 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a2c6ef-4b6d-4aa9-ac87-c837e05bd6c3 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:51.722669 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0680b3-be64-490a-a4c7-37c6c282744c None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:51.722795 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a2c6ef-4b6d-4aa9-ac87-c837e05bd6c3 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:51.722926 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0680b3-be64-490a-a4c7-37c6c282744c None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:51.723038 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a2c6ef-4b6d-4aa9-ac87-c837e05bd6c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:51.723158 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0680b3-be64-490a-a4c7-37c6c282744c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:51.745384 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-f9a2c6ef-4b6d-4aa9-ac87-c837e05bd6c3 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 19:57:51.745697 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-0c0680b3-be64-490a-a4c7-37c6c282744c None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 19:57:51.747352 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-0c0680b3-be64-490a-a4c7-37c6c282744c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 19:57:51.747518 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-f9a2c6ef-4b6d-4aa9-ac87-c837e05bd6c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 19:57:51.811211 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:57:51.847432 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 38/74] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 19:57:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 19:57:51.848429 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0680b3-be64-490a-a4c7-37c6c282744c None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:57:51.855466 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 37/75] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 19:57:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 19:57:51.856606 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a2c6ef-4b6d-4aa9-ac87-c837e05bd6c3 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:57:51.983469 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0680b3-be64-490a-a4c7-37c6c282744c None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:57:52.026647 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:57:52.029965 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6a9393-fc53-414c-a141-f3f298986f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:57:52.031640 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6a9393-fc53-414c-a141-f3f298986f2f None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:52.032029 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6a9393-fc53-414c-a141-f3f298986f2f None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:52.032647 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6a9393-fc53-414c-a141-f3f298986f2f None admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:52.033436 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6a9393-fc53-414c-a141-f3f298986f2f None admin] RBAC: Adding query filter params (name=member) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:57:52.033837 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6a9393-fc53-414c-a141-f3f298986f2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:57:52.035341 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6a9393-fc53-414c-a141-f3f298986f2f None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:57:52.036228 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a2c6ef-4b6d-4aa9-ac87-c837e05bd6c3 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:57:52.052206 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 39/76] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Sep 6 19:57:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:57:52.053544 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6a9393-fc53-414c-a141-f3f298986f2f None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:57:52.087982 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4285e715-a365-47ad-8480-963ef7bf582b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:57:52.088681 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4285e715-a365-47ad-8480-963ef7bf582b None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:52.088967 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4285e715-a365-47ad-8480-963ef7bf582b None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:52.089246 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4285e715-a365-47ad-8480-963ef7bf582b None admin] PATH_INFO: `/v3/users/1e7095a0137b45c984179dcf25de9362` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:52.091077 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4285e715-a365-47ad-8480-963ef7bf582b None admin] RBAC: Authorizing `identity:get_user(user_id=1e7095a0137b45c984179dcf25de9362)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:57:52.093278 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4285e715-a365-47ad-8480-963ef7bf582b None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:57:52.095964 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 38/77] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 6 19:57:51 2022] GET /identity/v3/users/1e7095a0137b45c984179dcf25de9362 => generated 247 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:57:52.096408 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4285e715-a365-47ad-8480-963ef7bf582b None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:57:52.105797 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:57:52.226675 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6a9393-fc53-414c-a141-f3f298986f2f None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:57:52.282418 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4285e715-a365-47ad-8480-963ef7bf582b None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:57:52.288102 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddc8714-7bfb-48a3-a625-1f1e2c196dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:57:52.288911 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddc8714-7bfb-48a3-a625-1f1e2c196dd6 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:52.289314 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddc8714-7bfb-48a3-a625-1f1e2c196dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:52.289759 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddc8714-7bfb-48a3-a625-1f1e2c196dd6 None admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:52.290644 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddc8714-7bfb-48a3-a625-1f1e2c196dd6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:57:52.291114 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddc8714-7bfb-48a3-a625-1f1e2c196dd6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:57:52.293080 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddc8714-7bfb-48a3-a625-1f1e2c196dd6 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:57:52.305951 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 40/78] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 6 19:57:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:57:52.306863 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddc8714-7bfb-48a3-a625-1f1e2c196dd6 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:57:52.336872 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9610594-7563-4a8d-b33d-659a7cd272a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:57:52.337639 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9610594-7563-4a8d-b33d-659a7cd272a8 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:52.338175 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9610594-7563-4a8d-b33d-659a7cd272a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:52.338552 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9610594-7563-4a8d-b33d-659a7cd272a8 None admin] PATH_INFO: `/v3/users/a729c9f9f70c41eab4ccf79f7ed5ff84` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:52.358684 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9610594-7563-4a8d-b33d-659a7cd272a8 None admin] RBAC: Authorizing `identity:get_user(user_id=a729c9f9f70c41eab4ccf79f7ed5ff84)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:57:52.360503 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9610594-7563-4a8d-b33d-659a7cd272a8 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:57:52.362522 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 39/79] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 6 19:57:52 2022] GET /identity/v3/users/a729c9f9f70c41eab4ccf79f7ed5ff84 => generated 275 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:57:52.363398 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9610594-7563-4a8d-b33d-659a7cd272a8 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:57:52.398283 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:57:52.448570 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:57:52.454987 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddc8714-7bfb-48a3-a625-1f1e2c196dd6 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:57:52.496591 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d447a0-e065-46ad-81df-99fd873a1f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:57:52.497222 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d447a0-e065-46ad-81df-99fd873a1f2f None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:52.497512 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d447a0-e065-46ad-81df-99fd873a1f2f None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:52.497818 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d447a0-e065-46ad-81df-99fd873a1f2f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:52.514715 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9610594-7563-4a8d-b33d-659a7cd272a8 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:57:52.558617 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdf547e-9f9e-4d7c-aa85-d223ad9e3d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:57:52.559548 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdf547e-9f9e-4d7c-aa85-d223ad9e3d58 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:52.559885 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdf547e-9f9e-4d7c-aa85-d223ad9e3d58 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:52.560307 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdf547e-9f9e-4d7c-aa85-d223ad9e3d58 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:52.561999 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdf547e-9f9e-4d7c-aa85-d223ad9e3d58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:57:52.563676 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdf547e-9f9e-4d7c-aa85-d223ad9e3d58 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:57:52.566001 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 40/80] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 19:57:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:57:52.568380 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdf547e-9f9e-4d7c-aa85-d223ad9e3d58 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:57:52.599067 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d447a0-e065-46ad-81df-99fd873a1f2f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:57:52.600534 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d447a0-e065-46ad-81df-99fd873a1f2f None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:57:52.675813 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:57:52.688001 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdf547e-9f9e-4d7c-aa85-d223ad9e3d58 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:57:52.722852 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:57:52.729166 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 41/81] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 6 19:57:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 19:57:52.730386 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d447a0-e065-46ad-81df-99fd873a1f2f None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:57:52.737559 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af175ded-b791-4af6-bec0-dcab64f8e911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:57:52.738639 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af175ded-b791-4af6-bec0-dcab64f8e911 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:52.739093 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af175ded-b791-4af6-bec0-dcab64f8e911 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:52.739545 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af175ded-b791-4af6-bec0-dcab64f8e911 None admin] PATH_INFO: `/v3/users/a729c9f9f70c41eab4ccf79f7ed5ff84` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:52.741516 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af175ded-b791-4af6-bec0-dcab64f8e911 None admin] RBAC: Authorizing `identity:get_user(user_id=a729c9f9f70c41eab4ccf79f7ed5ff84)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:57:52.744112 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af175ded-b791-4af6-bec0-dcab64f8e911 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:57:52.752932 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 41/82] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 6 19:57:52 2022] GET /identity/v3/users/a729c9f9f70c41eab4ccf79f7ed5ff84 => generated 275 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:57:52.752932 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af175ded-b791-4af6-bec0-dcab64f8e911 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:57:52.868306 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d447a0-e065-46ad-81df-99fd873a1f2f None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:57:52.893153 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af175ded-b791-4af6-bec0-dcab64f8e911 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:57:52.921148 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e926ddaf-b90f-4315-8501-8f0b8340f8db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:57:52.922154 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e926ddaf-b90f-4315-8501-8f0b8340f8db None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:52.922607 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e926ddaf-b90f-4315-8501-8f0b8340f8db None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:52.923022 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e926ddaf-b90f-4315-8501-8f0b8340f8db None admin] PATH_INFO: `/v3/roles/member` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:52.937130 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b782a103-aadf-414d-b1ec-e834b703c42e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:57:52.937893 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b782a103-aadf-414d-b1ec-e834b703c42e None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:52.938179 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b782a103-aadf-414d-b1ec-e834b703c42e None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:52.938462 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b782a103-aadf-414d-b1ec-e834b703c42e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:52.939323 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e926ddaf-b90f-4315-8501-8f0b8340f8db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 19:57:52.944456 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-e926ddaf-b90f-4315-8501-8f0b8340f8db None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 19:57:52.944456 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:57:52.944456 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:57:52.944456 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:57:52.944456 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:57:52.944456 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:57:52.944456 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:57:52.944456 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:57:52.944456 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:57:52.944456 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:57:52.944456 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:57:52.944456 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:57:52.944456 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 19:57:52.944456 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 19:57:52.944456 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 19:57:52.944456 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 19:57:52.945485 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 19:57:52.945485 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 19:57:52.945485 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 19:57:52.945485 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 19:57:52.945485 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:57:52.945485 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:57:52.945485 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:57:52.945485 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:57:52.945485 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:57:52.945485 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:57:52.945485 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:57:52.945485 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 19:57:52.945485 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:57:52.945485 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:57:52.945485 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:57:52.945485 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:57:52.946297 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:57:52.946297 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:57:52.946297 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:57:52.946297 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:57:52.946297 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 19:57:52.946297 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 19:57:52.946297 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 19:57:52.946297 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 19:57:52.946297 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 19:57:52.946297 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 19:57:52.946297 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 19:57:52.946297 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 19:57:52.946297 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 42/83] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 6 19:57:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:57:52.947513 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e926ddaf-b90f-4315-8501-8f0b8340f8db None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:57:52.973697 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:57:53.028682 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b782a103-aadf-414d-b1ec-e834b703c42e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:57:53.030876 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b782a103-aadf-414d-b1ec-e834b703c42e None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:57:53.031458 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:57:53.086653 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e926ddaf-b90f-4315-8501-8f0b8340f8db None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:57:53.128593 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6171531c-5690-48f3-bd0d-6be047e95fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:57:53.129128 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6171531c-5690-48f3-bd0d-6be047e95fc2 None admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:53.129412 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6171531c-5690-48f3-bd0d-6be047e95fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:53.129727 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6171531c-5690-48f3-bd0d-6be047e95fc2 None admin] PATH_INFO: `/v3/users` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:53.130401 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6171531c-5690-48f3-bd0d-6be047e95fc2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:57:53.131984 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6171531c-5690-48f3-bd0d-6be047e95fc2 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:57:53.161734 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 42/84] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 6 19:57:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 19:57:53.162753 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b782a103-aadf-414d-b1ec-e834b703c42e None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:57:53.163928 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 43/85] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Sep 6 19:57:52 2022] POST /identity/v3/users => generated 289 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 19:57:53.165134 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6171531c-5690-48f3-bd0d-6be047e95fc2 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:57:53.267165 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:57:53.318872 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6171531c-5690-48f3-bd0d-6be047e95fc2 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:57:53.338760 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:57:53.355516 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b782a103-aadf-414d-b1ec-e834b703c42e None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:57:53.371201 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef0c47f-df02-44c0-9636-028af72516b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:57:53.371851 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef0c47f-df02-44c0-9636-028af72516b3 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:53.372137 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef0c47f-df02-44c0-9636-028af72516b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:53.372414 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef0c47f-df02-44c0-9636-028af72516b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:53.415636 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa1be82-68de-4535-b3b8-a8e408f50d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:57:53.416645 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa1be82-68de-4535-b3b8-a8e408f50d82 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:53.417128 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa1be82-68de-4535-b3b8-a8e408f50d82 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:53.417600 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa1be82-68de-4535-b3b8-a8e408f50d82 None admin] PATH_INFO: `/v3/projects/197e55aa210c42abbf12e585a968f8c5` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:53.420400 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa1be82-68de-4535-b3b8-a8e408f50d82 None admin] RBAC: Authorizing `identity:get_project(project_id=197e55aa210c42abbf12e585a968f8c5)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:57:53.423208 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa1be82-68de-4535-b3b8-a8e408f50d82 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:57:53.426521 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 43/86] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 6 19:57:53 2022] GET /identity/v3/projects/197e55aa210c42abbf12e585a968f8c5 => generated 298 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:57:53.427722 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa1be82-68de-4535-b3b8-a8e408f50d82 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:57:53.464941 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef0c47f-df02-44c0-9636-028af72516b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:57:53.467360 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef0c47f-df02-44c0-9636-028af72516b3 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:57:53.616494 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa1be82-68de-4535-b3b8-a8e408f50d82 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:57:53.674091 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 44/87] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 6 19:57:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 509 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 19:57:53.676009 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef0c47f-df02-44c0-9636-028af72516b3 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:57:53.682182 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:57:53.705880 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f04f12-2f8b-4a1a-9a9c-7f3aa5717d90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:57:53.707122 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f04f12-2f8b-4a1a-9a9c-7f3aa5717d90 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:53.707797 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f04f12-2f8b-4a1a-9a9c-7f3aa5717d90 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:53.708867 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f04f12-2f8b-4a1a-9a9c-7f3aa5717d90 None admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:53.711062 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:57:53.711705 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f04f12-2f8b-4a1a-9a9c-7f3aa5717d90 None admin] RBAC: Adding query filter params (name=member) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:57:53.712498 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f04f12-2f8b-4a1a-9a9c-7f3aa5717d90 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:57:53.716716 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f04f12-2f8b-4a1a-9a9c-7f3aa5717d90 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:57:53.737741 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 44/88] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Sep 6 19:57:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:57:53.743990 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f04f12-2f8b-4a1a-9a9c-7f3aa5717d90 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:57:53.918158 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef0c47f-df02-44c0-9636-028af72516b3 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:57:54.004247 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d131a8-1627-4d6e-884c-04a6e87b4111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:57:54.005249 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d131a8-1627-4d6e-884c-04a6e87b4111 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:54.007110 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d131a8-1627-4d6e-884c-04a6e87b4111 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:54.007110 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d131a8-1627-4d6e-884c-04a6e87b4111 None admin] PATH_INFO: `/v3/projects/197e55aa210c42abbf12e585a968f8c5` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:54.008679 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d131a8-1627-4d6e-884c-04a6e87b4111 None admin] RBAC: Authorizing `identity:get_project(project_id=197e55aa210c42abbf12e585a968f8c5)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:57:54.011570 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f04f12-2f8b-4a1a-9a9c-7f3aa5717d90 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:57:54.012236 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d131a8-1627-4d6e-884c-04a6e87b4111 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:57:54.015990 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 45/89] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 6 19:57:53 2022] GET /identity/v3/projects/197e55aa210c42abbf12e585a968f8c5 => generated 298 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:57:54.018863 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d131a8-1627-4d6e-884c-04a6e87b4111 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:57:54.086983 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919cc7f9-a286-4077-9640-a7d0072352d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:57:54.088497 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919cc7f9-a286-4077-9640-a7d0072352d7 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:54.089329 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919cc7f9-a286-4077-9640-a7d0072352d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:54.090171 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919cc7f9-a286-4077-9640-a7d0072352d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:54.092377 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919cc7f9-a286-4077-9640-a7d0072352d7 None admin] RBAC: Adding query filter params (role.id=e0ab41d8ae22474688b5361bcc75e0f0, scope.project.id=197e55aa210c42abbf12e585a968f8c5, user.id=1e7095a0137b45c984179dcf25de9362) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:57:54.093314 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919cc7f9-a286-4077-9640-a7d0072352d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:57:54.096614 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919cc7f9-a286-4077-9640-a7d0072352d7 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:57:54.109489 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 45/90] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Sep 6 19:57:53 2022] GET /identity/v3/role_assignments?user.id=1e7095a0137b45c984179dcf25de9362&scope.project.id=197e55aa210c42abbf12e585a968f8c5&role.id=e0ab41d8ae22474688b5361bcc75e0f0 => generated 259 bytes in 371 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:57:54.111454 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919cc7f9-a286-4077-9640-a7d0072352d7 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:57:54.118247 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:57:54.142605 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d131a8-1627-4d6e-884c-04a6e87b4111 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:57:54.186037 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3013ef82-c283-4e97-8c6b-1a6c7bbf0f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:57:54.186851 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3013ef82-c283-4e97-8c6b-1a6c7bbf0f84 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:54.187236 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3013ef82-c283-4e97-8c6b-1a6c7bbf0f84 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:54.187626 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3013ef82-c283-4e97-8c6b-1a6c7bbf0f84 None admin] PATH_INFO: `/v3/projects/197e55aa210c42abbf12e585a968f8c5` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:54.188973 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3013ef82-c283-4e97-8c6b-1a6c7bbf0f84 None admin] RBAC: Authorizing `identity:get_project(project_id=197e55aa210c42abbf12e585a968f8c5)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:57:54.190740 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3013ef82-c283-4e97-8c6b-1a6c7bbf0f84 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:57:54.193235 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 46/91] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 6 19:57:54 2022] GET /identity/v3/projects/197e55aa210c42abbf12e585a968f8c5 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:57:54.193952 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3013ef82-c283-4e97-8c6b-1a6c7bbf0f84 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:57:54.235787 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:57:54.328112 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3013ef82-c283-4e97-8c6b-1a6c7bbf0f84 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:57:54.339843 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919cc7f9-a286-4077-9640-a7d0072352d7 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:57:54.370861 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128cacfd-02d9-411b-8742-f62d17e82c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:57:54.372025 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128cacfd-02d9-411b-8742-f62d17e82c7a None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:54.373070 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128cacfd-02d9-411b-8742-f62d17e82c7a None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:54.373552 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128cacfd-02d9-411b-8742-f62d17e82c7a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:54.374692 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128cacfd-02d9-411b-8742-f62d17e82c7a None admin] RBAC: Adding query filter params (role.id=4eea39e24b06431aa357053fe7907a3d, scope.project.id=197e55aa210c42abbf12e585a968f8c5, user.id=a729c9f9f70c41eab4ccf79f7ed5ff84) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:57:54.375307 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128cacfd-02d9-411b-8742-f62d17e82c7a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:57:54.377216 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128cacfd-02d9-411b-8742-f62d17e82c7a None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:57:54.386045 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 47/92] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Sep 6 19:57:54 2022] GET /identity/v3/role_assignments?user.id=a729c9f9f70c41eab4ccf79f7ed5ff84&scope.project.id=197e55aa210c42abbf12e585a968f8c5&role.id=4eea39e24b06431aa357053fe7907a3d => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:57:54.393660 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128cacfd-02d9-411b-8742-f62d17e82c7a None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:57:54.437301 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:57:54.469643 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f992f7-260a-410e-885e-2f2770101135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:57:54.469643 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f992f7-260a-410e-885e-2f2770101135 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:54.469963 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f992f7-260a-410e-885e-2f2770101135 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:54.470671 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f992f7-260a-410e-885e-2f2770101135 None admin] PATH_INFO: `/v3/users/a729c9f9f70c41eab4ccf79f7ed5ff84` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:54.474196 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f992f7-260a-410e-885e-2f2770101135 None admin] RBAC: Authorizing `identity:get_user(user_id=a729c9f9f70c41eab4ccf79f7ed5ff84)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:57:54.476345 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f992f7-260a-410e-885e-2f2770101135 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:57:54.479404 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 46/93] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 6 19:57:54 2022] GET /identity/v3/users/a729c9f9f70c41eab4ccf79f7ed5ff84 => generated 275 bytes in 369 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:57:54.487419 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f992f7-260a-410e-885e-2f2770101135 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:57:54.733347 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128cacfd-02d9-411b-8742-f62d17e82c7a None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:57:54.814167 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f992f7-260a-410e-885e-2f2770101135 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:57:54.854211 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:57:54.858458 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eaa5de-5e93-4e35-be00-b807eae5b137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:57:54.859438 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eaa5de-5e93-4e35-be00-b807eae5b137 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:54.859873 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eaa5de-5e93-4e35-be00-b807eae5b137 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:54.860358 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eaa5de-5e93-4e35-be00-b807eae5b137 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:54.861738 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04eaa5de-5e93-4e35-be00-b807eae5b137 None admin] RBAC: Adding query filter params (role.id=b02170460f054c259e825ab6e35f8537, scope.project.id=197e55aa210c42abbf12e585a968f8c5, user.id=a729c9f9f70c41eab4ccf79f7ed5ff84) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:57:54.862252 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04eaa5de-5e93-4e35-be00-b807eae5b137 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:57:54.864677 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04eaa5de-5e93-4e35-be00-b807eae5b137 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:57:54.875183 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 48/94] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Sep 6 19:57:54 2022] GET /identity/v3/role_assignments?user.id=a729c9f9f70c41eab4ccf79f7ed5ff84&scope.project.id=197e55aa210c42abbf12e585a968f8c5&role.id=b02170460f054c259e825ab6e35f8537 => generated 259 bytes in 488 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:57:54.883973 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449b85dd-79d9-497d-ab02-cefe81c66b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:57:54.884980 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449b85dd-79d9-497d-ab02-cefe81c66b8e None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:54.885312 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449b85dd-79d9-497d-ab02-cefe81c66b8e None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:54.885649 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449b85dd-79d9-497d-ab02-cefe81c66b8e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:55.017736 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449b85dd-79d9-497d-ab02-cefe81c66b8e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:57:55.019299 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449b85dd-79d9-497d-ab02-cefe81c66b8e None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:57:55.192467 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 47/95] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 6 19:57:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 706 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 19:57:55.198727 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eaa5de-5e93-4e35-be00-b807eae5b137 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:57:55.318273 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:57:55.350849 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eaa5de-5e93-4e35-be00-b807eae5b137 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:57:55.395914 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d776df31-e69d-4b8b-902d-e8dca49f7c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:57:55.396894 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d776df31-e69d-4b8b-902d-e8dca49f7c93 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:55.397189 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d776df31-e69d-4b8b-902d-e8dca49f7c93 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:55.397515 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d776df31-e69d-4b8b-902d-e8dca49f7c93 None admin] PATH_INFO: `/v3/projects/183ec5fbd9a04da1ab46d9eaccdf955a` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:55.398856 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d776df31-e69d-4b8b-902d-e8dca49f7c93 None admin] RBAC: Authorizing `identity:get_project(project_id=183ec5fbd9a04da1ab46d9eaccdf955a)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:57:55.400393 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d776df31-e69d-4b8b-902d-e8dca49f7c93 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:57:55.402201 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 49/96] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 6 19:57:55 2022] GET /identity/v3/projects/183ec5fbd9a04da1ab46d9eaccdf955a => generated 312 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:57:55.408701 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449b85dd-79d9-497d-ab02-cefe81c66b8e None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:57:55.437370 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:57:55.550003 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449b85dd-79d9-497d-ab02-cefe81c66b8e None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:57:55.610062 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42f856a-ae3d-4978-a99a-1ebbff780cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:57:55.611222 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42f856a-ae3d-4978-a99a-1ebbff780cd8 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:55.611772 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42f856a-ae3d-4978-a99a-1ebbff780cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:55.612287 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42f856a-ae3d-4978-a99a-1ebbff780cd8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:55.618033 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42f856a-ae3d-4978-a99a-1ebbff780cd8 None admin] RBAC: Adding query filter params (role.id=4eea39e24b06431aa357053fe7907a3d, scope.project.id=183ec5fbd9a04da1ab46d9eaccdf955a, user.id=a729c9f9f70c41eab4ccf79f7ed5ff84) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:57:55.618738 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42f856a-ae3d-4978-a99a-1ebbff780cd8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:57:55.626046 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42f856a-ae3d-4978-a99a-1ebbff780cd8 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:57:55.644436 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 48/97] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Sep 6 19:57:55 2022] GET /identity/v3/role_assignments?user.id=a729c9f9f70c41eab4ccf79f7ed5ff84&scope.project.id=183ec5fbd9a04da1ab46d9eaccdf955a&role.id=4eea39e24b06431aa357053fe7907a3d => generated 259 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:57:56.779509 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299e3511-9d30-4989-b543-defb065d78c7 None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:56.780090 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299e3511-9d30-4989-b543-defb065d78c7 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:56.780090 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299e3511-9d30-4989-b543-defb065d78c7 None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:56.781639 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 50/98] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 19:57:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 19:57:56.787636 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a421990-cff5-45d3-a3e8-767499faafe9 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:56.788153 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a421990-cff5-45d3-a3e8-767499faafe9 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:56.788367 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a421990-cff5-45d3-a3e8-767499faafe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:56.817488 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-7a421990-cff5-45d3-a3e8-767499faafe9 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 19:57:56.818784 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-7a421990-cff5-45d3-a3e8-767499faafe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 19:57:56.942373 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 49/99] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 19:57:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 19:57:57.579709 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299e3511-9d30-4989-b543-defb065d78c7 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:57:57.763197 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299e3511-9d30-4989-b543-defb065d78c7 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:57:57.827478 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d6cd6d-b7f9-4140-b3a6-ec2fdebdbb1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:57:57.828980 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d6cd6d-b7f9-4140-b3a6-ec2fdebdbb1a None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:57.829531 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d6cd6d-b7f9-4140-b3a6-ec2fdebdbb1a None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:57.830053 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d6cd6d-b7f9-4140-b3a6-ec2fdebdbb1a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:57.831608 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d6cd6d-b7f9-4140-b3a6-ec2fdebdbb1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:57:57.833700 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d6cd6d-b7f9-4140-b3a6-ec2fdebdbb1a None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:57:57.836065 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 51/100] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 19:57:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:57:57.851600 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a421990-cff5-45d3-a3e8-767499faafe9 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:57:57.891857 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:57:58.089650 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a421990-cff5-45d3-a3e8-767499faafe9 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:57:58.165403 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191de79c-8bbe-40c1-a6d5-d528199da7e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:57:58.166086 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191de79c-8bbe-40c1-a6d5-d528199da7e2 None admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:58.166379 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191de79c-8bbe-40c1-a6d5-d528199da7e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:58.166722 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191de79c-8bbe-40c1-a6d5-d528199da7e2 None admin] PATH_INFO: `/v3/projects` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:58.167425 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191de79c-8bbe-40c1-a6d5-d528199da7e2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:57:58.168940 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191de79c-8bbe-40c1-a6d5-d528199da7e2 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:57:58.205496 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 50/101] 10.222.0.20 () {60 vars in 1141 bytes} [Tue Sep 6 19:57:57 2022] POST /identity/v3/projects => generated 302 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 19:57:58.276058 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beb43d2-043f-4e6a-856b-7b8085f536ad None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:58.276718 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beb43d2-043f-4e6a-856b-7b8085f536ad None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:58.277474 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beb43d2-043f-4e6a-856b-7b8085f536ad None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:58.280537 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 52/102] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 19:57:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 19:57:58.289294 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58eab0f-8fff-48fc-8f17-9a1edbf98cb8 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:58.289682 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58eab0f-8fff-48fc-8f17-9a1edbf98cb8 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:58.290025 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58eab0f-8fff-48fc-8f17-9a1edbf98cb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:58.316232 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-a58eab0f-8fff-48fc-8f17-9a1edbf98cb8 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 19:57:58.323374 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-a58eab0f-8fff-48fc-8f17-9a1edbf98cb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 19:57:58.391842 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3128de13-6e53-44a3-b02a-e040b7284188 None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:58.392482 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3128de13-6e53-44a3-b02a-e040b7284188 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:58.392960 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3128de13-6e53-44a3-b02a-e040b7284188 None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:58.395371 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 53/103] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 19:57:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 19:57:58.401415 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb299028-7984-4a68-a3e9-d714609db7e0 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:58.402066 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb299028-7984-4a68-a3e9-d714609db7e0 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:58.402561 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb299028-7984-4a68-a3e9-d714609db7e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:58.422035 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:57:58.444299 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 51/104] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 19:57:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 19:57:58.454919 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-bb299028-7984-4a68-a3e9-d714609db7e0 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 19:57:58.456775 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-bb299028-7984-4a68-a3e9-d714609db7e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 19:57:58.491502 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:57:58.582468 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 54/105] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 19:57:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 19:57:58.734969 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd4ac0a-cddf-49f1-af6c-812df323e837 None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:58.735304 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd4ac0a-cddf-49f1-af6c-812df323e837 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:58.735492 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd4ac0a-cddf-49f1-af6c-812df323e837 None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:58.737347 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 52/106] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 19:57:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 19:57:58.744975 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f4bbfa-1818-4306-8acc-bad4c61a3594 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:58.745633 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f4bbfa-1818-4306-8acc-bad4c61a3594 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:58.746132 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f4bbfa-1818-4306-8acc-bad4c61a3594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:58.774868 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-81f4bbfa-1818-4306-8acc-bad4c61a3594 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 19:57:58.777295 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-81f4bbfa-1818-4306-8acc-bad4c61a3594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 19:57:58.875660 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 55/107] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 19:57:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 19:57:58.880815 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbde9a0-d1f0-4e7a-b74f-ad30817061c3 None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:58.881317 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbde9a0-d1f0-4e7a-b74f-ad30817061c3 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:58.881390 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbde9a0-d1f0-4e7a-b74f-ad30817061c3 None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:58.883078 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 53/108] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 19:57:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 19:57:58.890668 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afaba2b-1d28-4b57-8dc4-b2ede65dc68a None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:58.890993 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afaba2b-1d28-4b57-8dc4-b2ede65dc68a None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:58.891465 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afaba2b-1d28-4b57-8dc4-b2ede65dc68a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:58.913631 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-6afaba2b-1d28-4b57-8dc4-b2ede65dc68a None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 19:57:58.915201 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-6afaba2b-1d28-4b57-8dc4-b2ede65dc68a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 19:57:59.040109 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbde9a0-d1f0-4e7a-b74f-ad30817061c3 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:57:59.049945 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 56/109] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 19:57:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 19:57:59.099961 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afaba2b-1d28-4b57-8dc4-b2ede65dc68a None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:57:59.104743 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:57:59.417191 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbde9a0-d1f0-4e7a-b74f-ad30817061c3 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:57:59.504989 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afaba2b-1d28-4b57-8dc4-b2ede65dc68a None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:57:59.519962 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a126b8-1542-4307-906c-c9e94aee0dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:57:59.520639 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a126b8-1542-4307-906c-c9e94aee0dd1 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:59.520922 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a126b8-1542-4307-906c-c9e94aee0dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:59.521201 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a126b8-1542-4307-906c-c9e94aee0dd1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:59.561814 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a126b8-1542-4307-906c-c9e94aee0dd1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 19:57:59.566990 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-b4a126b8-1542-4307-906c-c9e94aee0dd1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 19:57:59.566990 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:57:59.566990 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:57:59.566990 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:57:59.566990 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:57:59.566990 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:57:59.566990 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:57:59.566990 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:57:59.566990 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:57:59.566990 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:57:59.566990 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:57:59.566990 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:57:59.566990 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 19:57:59.566990 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 19:57:59.566990 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 19:57:59.566990 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 19:57:59.568185 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 19:57:59.568185 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 19:57:59.568185 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 19:57:59.568185 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 19:57:59.568185 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:57:59.568185 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:57:59.568185 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:57:59.568185 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:57:59.568185 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:57:59.568185 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:57:59.568185 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:57:59.568185 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 19:57:59.568185 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:57:59.568185 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:57:59.568185 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:57:59.568185 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:57:59.568935 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:57:59.568935 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:57:59.568935 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:57:59.568935 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:57:59.568935 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 19:57:59.568935 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 19:57:59.568935 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 19:57:59.568935 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 19:57:59.568935 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 19:57:59.568935 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 19:57:59.568935 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 19:57:59.568935 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 19:57:59.571665 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 54/110] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 6 19:57:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 532 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 19:57:59.572966 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a126b8-1542-4307-906c-c9e94aee0dd1 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:57:59.591403 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e719ad3-e50c-4392-a59a-036f914adcac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:57:59.592259 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e719ad3-e50c-4392-a59a-036f914adcac None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:59.592592 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e719ad3-e50c-4392-a59a-036f914adcac None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:59.592874 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e719ad3-e50c-4392-a59a-036f914adcac None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:59.611036 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e719ad3-e50c-4392-a59a-036f914adcac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 19:57:59.616362 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-6e719ad3-e50c-4392-a59a-036f914adcac None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 19:57:59.616362 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:57:59.616362 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:57:59.616362 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:57:59.616362 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:57:59.616362 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:57:59.616362 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:57:59.616362 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:57:59.616362 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:57:59.616362 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:57:59.616362 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:57:59.616362 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:57:59.616362 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 19:57:59.616362 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 19:57:59.616362 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 19:57:59.616362 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 19:57:59.617279 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 19:57:59.617279 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 19:57:59.617279 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 19:57:59.617279 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 19:57:59.617279 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:57:59.617279 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:57:59.617279 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:57:59.617279 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:57:59.617279 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:57:59.617279 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:57:59.617279 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:57:59.617279 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 19:57:59.617279 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:57:59.617279 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:57:59.617279 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:57:59.617279 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:57:59.618104 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:57:59.618104 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:57:59.618104 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:57:59.618104 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:57:59.618104 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 19:57:59.618104 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 19:57:59.618104 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 19:57:59.618104 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 19:57:59.618104 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 19:57:59.618104 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 19:57:59.618104 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 19:57:59.618104 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 19:57:59.620795 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 57/111] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 6 19:57:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 521 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 19:57:59.621974 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e719ad3-e50c-4392-a59a-036f914adcac None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:57:59.676056 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:57:59.725909 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:57:59.772282 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a126b8-1542-4307-906c-c9e94aee0dd1 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:57:59.796898 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e719ad3-e50c-4392-a59a-036f914adcac None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:57:59.846140 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ecd133-6721-40e7-8b9a-bc295251c5a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:57:59.847335 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ecd133-6721-40e7-8b9a-bc295251c5a4 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:59.847910 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ecd133-6721-40e7-8b9a-bc295251c5a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:59.848393 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ecd133-6721-40e7-8b9a-bc295251c5a4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:59.861677 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a54d8a-b384-4688-8357-e3394fff2efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:57:59.862509 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a54d8a-b384-4688-8357-e3394fff2efe None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:59.862957 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a54d8a-b384-4688-8357-e3394fff2efe None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:59.863337 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a54d8a-b384-4688-8357-e3394fff2efe None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:59.866378 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ecd133-6721-40e7-8b9a-bc295251c5a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 19:57:59.872481 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-29ecd133-6721-40e7-8b9a-bc295251c5a4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 19:57:59.872481 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:57:59.872481 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:57:59.872481 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:57:59.872481 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:57:59.872481 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:57:59.872481 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:57:59.872481 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:57:59.872481 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:57:59.872481 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:57:59.872481 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:57:59.872481 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:57:59.872481 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 19:57:59.872481 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 19:57:59.872481 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 19:57:59.872481 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 19:57:59.873588 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 19:57:59.873588 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 19:57:59.873588 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 19:57:59.873588 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 19:57:59.873588 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:57:59.873588 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:57:59.873588 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:57:59.873588 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:57:59.873588 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:57:59.873588 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:57:59.873588 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:57:59.873588 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 19:57:59.873588 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:57:59.873588 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:57:59.873588 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:57:59.873588 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:57:59.874425 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:57:59.874425 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:57:59.874425 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:57:59.874425 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:57:59.874425 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 19:57:59.874425 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 19:57:59.874425 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 19:57:59.874425 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 19:57:59.874425 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 19:57:59.874425 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 19:57:59.874425 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 19:57:59.874425 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 19:57:59.877665 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a54d8a-b384-4688-8357-e3394fff2efe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 19:57:59.881795 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 58/112] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Sep 6 19:57:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:57:59.883348 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bdf4e8-991c-45bc-ad59-c57243a4f136 None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:57:59.883643 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bdf4e8-991c-45bc-ad59-c57243a4f136 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:57:59.883927 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bdf4e8-991c-45bc-ad59-c57243a4f136 None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:57:59.884162 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-45a54d8a-b384-4688-8357-e3394fff2efe None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 19:57:59.884162 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:57:59.884162 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:57:59.884162 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:57:59.884162 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:57:59.884162 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:57:59.884162 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:57:59.884162 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:57:59.884162 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:57:59.884162 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:57:59.884162 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:57:59.884162 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:57:59.884162 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 19:57:59.884162 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 19:57:59.884162 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 19:57:59.884162 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 19:57:59.890672 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 19:57:59.890672 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 19:57:59.890672 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 19:57:59.890672 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 19:57:59.890672 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:57:59.890672 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:57:59.890672 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:57:59.890672 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:57:59.890672 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:57:59.890672 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:57:59.890672 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:57:59.890672 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 19:57:59.890672 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:57:59.890672 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:57:59.890672 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:57:59.890672 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:57:59.891811 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:57:59.891811 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:57:59.891811 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:57:59.891811 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:57:59.891811 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 19:57:59.891811 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 19:57:59.891811 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 19:57:59.891811 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 19:57:59.891811 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 19:57:59.891811 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 19:57:59.891811 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 19:57:59.891811 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 19:57:59.891811 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 55/113] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Sep 6 19:57:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:57:59.891811 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a54d8a-b384-4688-8357-e3394fff2efe None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:57:59.892682 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 59/114] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 19:57:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 19:57:59.892682 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bdf4e8-991c-45bc-ad59-c57243a4f136 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:00.057509 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bdf4e8-991c-45bc-ad59-c57243a4f136 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:00.058061 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a54d8a-b384-4688-8357-e3394fff2efe None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:00.081157 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:00.099247 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:00.105869 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aaaf29-aaf9-4609-be03-a29e154ebcd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:00.106756 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aaaf29-aaf9-4609-be03-a29e154ebcd3 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:00.107050 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aaaf29-aaf9-4609-be03-a29e154ebcd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:00.107328 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aaaf29-aaf9-4609-be03-a29e154ebcd3 None admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:00.108035 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19aaaf29-aaf9-4609-be03-a29e154ebcd3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:58:00.108349 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19aaaf29-aaf9-4609-be03-a29e154ebcd3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:00.109867 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19aaaf29-aaf9-4609-be03-a29e154ebcd3 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:00.123909 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd1b831-b3f9-4c08-acf6-472ce5654402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:00.124884 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd1b831-b3f9-4c08-acf6-472ce5654402 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:00.125340 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd1b831-b3f9-4c08-acf6-472ce5654402 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:00.125797 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd1b831-b3f9-4c08-acf6-472ce5654402 None admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:00.125975 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 56/115] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Sep 6 19:57:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:00.127010 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd1b831-b3f9-4c08-acf6-472ce5654402 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:58:00.127230 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aaaf29-aaf9-4609-be03-a29e154ebcd3 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:00.127727 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd1b831-b3f9-4c08-acf6-472ce5654402 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:00.129333 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd1b831-b3f9-4c08-acf6-472ce5654402 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:00.141499 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 60/116] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Sep 6 19:57:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:00.143131 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01da8d26-6e4e-49dc-9533-eb8f3bc802de None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:00.143566 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01da8d26-6e4e-49dc-9533-eb8f3bc802de None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:00.144227 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01da8d26-6e4e-49dc-9533-eb8f3bc802de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:00.174977 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-01da8d26-6e4e-49dc-9533-eb8f3bc802de None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 19:58:00.177301 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-01da8d26-6e4e-49dc-9533-eb8f3bc802de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 19:58:00.286084 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aaaf29-aaf9-4609-be03-a29e154ebcd3 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:00.317748 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 61/117] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 19:58:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 19:58:00.319087 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01da8d26-6e4e-49dc-9533-eb8f3bc802de None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:00.336543 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40fea88-6df0-4bf3-885c-61a7ddc8ac17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:00.338247 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40fea88-6df0-4bf3-885c-61a7ddc8ac17 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:00.338719 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40fea88-6df0-4bf3-885c-61a7ddc8ac17 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:00.339226 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40fea88-6df0-4bf3-885c-61a7ddc8ac17 None admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:00.339982 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40fea88-6df0-4bf3-885c-61a7ddc8ac17 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:58:00.340290 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40fea88-6df0-4bf3-885c-61a7ddc8ac17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:00.342041 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40fea88-6df0-4bf3-885c-61a7ddc8ac17 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:00.359642 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 57/118] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 6 19:58:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:00.359642 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40fea88-6df0-4bf3-885c-61a7ddc8ac17 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:00.441782 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:00.503209 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01da8d26-6e4e-49dc-9533-eb8f3bc802de None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:00.540876 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:00.566012 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40fea88-6df0-4bf3-885c-61a7ddc8ac17 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:00.572833 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656f1406-0257-4cea-bfff-cb539cddb7d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:00.573321 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656f1406-0257-4cea-bfff-cb539cddb7d5 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:00.573628 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656f1406-0257-4cea-bfff-cb539cddb7d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:00.573897 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656f1406-0257-4cea-bfff-cb539cddb7d5 None admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:00.574720 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656f1406-0257-4cea-bfff-cb539cddb7d5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:58:00.575031 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656f1406-0257-4cea-bfff-cb539cddb7d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:00.576754 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656f1406-0257-4cea-bfff-cb539cddb7d5 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:00.595724 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 62/119] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 6 19:58:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:00.595724 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656f1406-0257-4cea-bfff-cb539cddb7d5 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:00.649382 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2b9954-6abf-4388-8875-9b0f6ebd2291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:00.651090 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2b9954-6abf-4388-8875-9b0f6ebd2291 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:00.651869 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2b9954-6abf-4388-8875-9b0f6ebd2291 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:00.652669 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2b9954-6abf-4388-8875-9b0f6ebd2291 None admin] PATH_INFO: `/v3/users/a729c9f9f70c41eab4ccf79f7ed5ff84` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:00.655191 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2b9954-6abf-4388-8875-9b0f6ebd2291 None admin] RBAC: Authorizing `identity:get_user(user_id=a729c9f9f70c41eab4ccf79f7ed5ff84)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:00.658517 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2b9954-6abf-4388-8875-9b0f6ebd2291 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:00.662223 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 58/120] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 6 19:58:00 2022] GET /identity/v3/users/a729c9f9f70c41eab4ccf79f7ed5ff84 => generated 275 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:00.671685 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2b9954-6abf-4388-8875-9b0f6ebd2291 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:00.735705 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656f1406-0257-4cea-bfff-cb539cddb7d5 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:00.781860 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97d847c-6a8e-425c-8713-9de18450fc28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:00.782727 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97d847c-6a8e-425c-8713-9de18450fc28 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:00.783149 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97d847c-6a8e-425c-8713-9de18450fc28 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:00.783553 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97d847c-6a8e-425c-8713-9de18450fc28 None admin] PATH_INFO: `/v3/users/37e092fe6c824b3cb149b714145f1710` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:00.801802 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97d847c-6a8e-425c-8713-9de18450fc28 None admin] RBAC: Authorizing `identity:get_user(user_id=37e092fe6c824b3cb149b714145f1710)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:00.803436 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97d847c-6a8e-425c-8713-9de18450fc28 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:00.805244 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 63/121] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 6 19:58:00 2022] GET /identity/v3/users/37e092fe6c824b3cb149b714145f1710 => generated 289 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:00.806327 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97d847c-6a8e-425c-8713-9de18450fc28 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:00.819660 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2b9954-6abf-4388-8875-9b0f6ebd2291 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:00.835990 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:00.858833 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:00.879056 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3f3e24-d34c-46bf-9841-da8588f49d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:00.880192 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3f3e24-d34c-46bf-9841-da8588f49d8a None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:00.880653 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3f3e24-d34c-46bf-9841-da8588f49d8a None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:00.881124 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3f3e24-d34c-46bf-9841-da8588f49d8a None admin] PATH_INFO: `/v3/users/1e7095a0137b45c984179dcf25de9362` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:00.883147 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3f3e24-d34c-46bf-9841-da8588f49d8a None admin] RBAC: Authorizing `identity:get_user(user_id=1e7095a0137b45c984179dcf25de9362)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:00.885753 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3f3e24-d34c-46bf-9841-da8588f49d8a None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:00.888687 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 59/122] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 6 19:58:00 2022] GET /identity/v3/users/1e7095a0137b45c984179dcf25de9362 => generated 247 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:00.889718 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3f3e24-d34c-46bf-9841-da8588f49d8a None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:01.009219 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97d847c-6a8e-425c-8713-9de18450fc28 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:01.064234 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3f3e24-d34c-46bf-9841-da8588f49d8a None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:01.083838 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba94227-56e3-450a-9ac7-b2223b0a3bfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:01.084976 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba94227-56e3-450a-9ac7-b2223b0a3bfa None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:01.085482 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba94227-56e3-450a-9ac7-b2223b0a3bfa None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:01.086100 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba94227-56e3-450a-9ac7-b2223b0a3bfa None admin] PATH_INFO: `/v3/users/a729c9f9f70c41eab4ccf79f7ed5ff84` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:01.088741 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba94227-56e3-450a-9ac7-b2223b0a3bfa None admin] RBAC: Authorizing `identity:get_user(user_id=a729c9f9f70c41eab4ccf79f7ed5ff84)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:01.091581 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba94227-56e3-450a-9ac7-b2223b0a3bfa None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:01.101589 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 64/123] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 6 19:58:00 2022] GET /identity/v3/users/a729c9f9f70c41eab4ccf79f7ed5ff84 => generated 275 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:01.101589 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba94227-56e3-450a-9ac7-b2223b0a3bfa None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:01.126550 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0920b0bd-87e2-47c4-a885-6f4c8f3c7a75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:01.127636 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0920b0bd-87e2-47c4-a885-6f4c8f3c7a75 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:01.128139 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0920b0bd-87e2-47c4-a885-6f4c8f3c7a75 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:01.128628 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0920b0bd-87e2-47c4-a885-6f4c8f3c7a75 None admin] PATH_INFO: `/v3/projects/197e55aa210c42abbf12e585a968f8c5` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:01.131091 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0920b0bd-87e2-47c4-a885-6f4c8f3c7a75 None admin] RBAC: Authorizing `identity:get_project(project_id=197e55aa210c42abbf12e585a968f8c5)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:01.134192 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0920b0bd-87e2-47c4-a885-6f4c8f3c7a75 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:01.142460 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 60/124] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 6 19:58:00 2022] GET /identity/v3/projects/197e55aa210c42abbf12e585a968f8c5 => generated 298 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:01.142460 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0920b0bd-87e2-47c4-a885-6f4c8f3c7a75 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:01.218689 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:01.292406 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba94227-56e3-450a-9ac7-b2223b0a3bfa None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:01.297528 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:01.331110 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0920b0bd-87e2-47c4-a885-6f4c8f3c7a75 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:01.362812 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9867fb-606e-4484-8508-2baf843b0573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:01.363584 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9867fb-606e-4484-8508-2baf843b0573 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:01.363868 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9867fb-606e-4484-8508-2baf843b0573 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:01.364442 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9867fb-606e-4484-8508-2baf843b0573 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:01.383376 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9867fb-606e-4484-8508-2baf843b0573 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 19:58:01.387657 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ea243d-9a1c-4082-805e-7e6da5db8d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:01.388771 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ea243d-9a1c-4082-805e-7e6da5db8d25 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:01.389269 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ea243d-9a1c-4082-805e-7e6da5db8d25 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:01.389433 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-5a9867fb-606e-4484-8508-2baf843b0573 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 19:58:01.389433 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:58:01.389433 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:58:01.389433 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:58:01.389433 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:58:01.389433 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:58:01.389433 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:58:01.389433 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:58:01.389433 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:58:01.389433 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:58:01.389433 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:58:01.389433 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:58:01.389433 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 19:58:01.389433 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 19:58:01.389433 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 19:58:01.389433 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 19:58:01.391282 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 19:58:01.391282 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 19:58:01.391282 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 19:58:01.391282 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 19:58:01.391282 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:58:01.391282 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:58:01.391282 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:58:01.391282 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:58:01.391282 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:58:01.391282 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:58:01.391282 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:58:01.391282 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 19:58:01.391282 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:58:01.391282 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:58:01.391282 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:58:01.391282 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:58:01.392835 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:58:01.392835 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:58:01.392835 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:58:01.392835 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:58:01.392835 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 19:58:01.392835 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 19:58:01.392835 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 19:58:01.392835 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 19:58:01.392835 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 19:58:01.392835 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 19:58:01.392835 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 19:58:01.392835 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 19:58:01.395869 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ea243d-9a1c-4082-805e-7e6da5db8d25 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:01.396169 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 65/125] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 6 19:58:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:58:01.396772 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9867fb-606e-4484-8508-2baf843b0573 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:01.538242 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ea243d-9a1c-4082-805e-7e6da5db8d25 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:01.541764 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ea243d-9a1c-4082-805e-7e6da5db8d25 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:01.558598 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9867fb-606e-4484-8508-2baf843b0573 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:01.622688 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:01.631601 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b426bb3-46cc-4af7-bb1f-604cfa27964c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:01.632714 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b426bb3-46cc-4af7-bb1f-604cfa27964c None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:01.633214 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b426bb3-46cc-4af7-bb1f-604cfa27964c None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:01.633830 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b426bb3-46cc-4af7-bb1f-604cfa27964c None admin] PATH_INFO: `/v3/projects/197e55aa210c42abbf12e585a968f8c5` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:01.636185 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b426bb3-46cc-4af7-bb1f-604cfa27964c None admin] RBAC: Authorizing `identity:get_project(project_id=197e55aa210c42abbf12e585a968f8c5)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:01.638891 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b426bb3-46cc-4af7-bb1f-604cfa27964c None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:01.643835 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 66/126] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 6 19:58:01 2022] GET /identity/v3/projects/197e55aa210c42abbf12e585a968f8c5 => generated 298 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:01.644924 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b426bb3-46cc-4af7-bb1f-604cfa27964c None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:01.706402 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:01.739274 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 61/127] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 6 19:58:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 601 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 19:58:01.741155 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ea243d-9a1c-4082-805e-7e6da5db8d25 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:01.850394 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b426bb3-46cc-4af7-bb1f-604cfa27964c None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:01.899623 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ea243d-9a1c-4082-805e-7e6da5db8d25 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:01.900378 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e537a5-eea2-4fb2-a9f1-a5d4a5373db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:01.900877 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e537a5-eea2-4fb2-a9f1-a5d4a5373db7 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:01.901301 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e537a5-eea2-4fb2-a9f1-a5d4a5373db7 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:01.901751 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e537a5-eea2-4fb2-a9f1-a5d4a5373db7 None admin] PATH_INFO: `/v3/projects/197e55aa210c42abbf12e585a968f8c5` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:01.903132 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e537a5-eea2-4fb2-a9f1-a5d4a5373db7 None admin] RBAC: Authorizing `identity:get_project(project_id=197e55aa210c42abbf12e585a968f8c5)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:01.904785 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e537a5-eea2-4fb2-a9f1-a5d4a5373db7 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:01.906683 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 67/128] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 6 19:58:01 2022] GET /identity/v3/projects/197e55aa210c42abbf12e585a968f8c5 => generated 298 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:01.907724 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e537a5-eea2-4fb2-a9f1-a5d4a5373db7 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:01.951109 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b472ffce-13bd-4d7b-9a4d-ee4ab33c55d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:01.952247 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b472ffce-13bd-4d7b-9a4d-ee4ab33c55d1 None admin] REQUEST_METHOD: `PUT` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:01.952712 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b472ffce-13bd-4d7b-9a4d-ee4ab33c55d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:01.953187 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b472ffce-13bd-4d7b-9a4d-ee4ab33c55d1 None admin] PATH_INFO: `/v3/projects/197e55aa210c42abbf12e585a968f8c5/users/a729c9f9f70c41eab4ccf79f7ed5ff84/roles/4eea39e24b06431aa357053fe7907a3d` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:01.956433 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b472ffce-13bd-4d7b-9a4d-ee4ab33c55d1 None admin] RBAC: Authorizing `identity:create_grant(project_id=197e55aa210c42abbf12e585a968f8c5, user_id=a729c9f9f70c41eab4ccf79f7ed5ff84, role_id=4eea39e24b06431aa357053fe7907a3d)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:01.958008 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b472ffce-13bd-4d7b-9a4d-ee4ab33c55d1 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:01.979172 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 62/129] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Sep 6 19:58:01 2022] PUT /identity/v3/projects/197e55aa210c42abbf12e585a968f8c5/users/a729c9f9f70c41eab4ccf79f7ed5ff84/roles/4eea39e24b06431aa357053fe7907a3d => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 19:58:01.980564 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b472ffce-13bd-4d7b-9a4d-ee4ab33c55d1 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:01.995458 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:02.046566 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:02.051091 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e537a5-eea2-4fb2-a9f1-a5d4a5373db7 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:02.107799 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01659466-9792-4dce-a9c4-b1976986ef50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:02.108458 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01659466-9792-4dce-a9c4-b1976986ef50 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:02.108743 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01659466-9792-4dce-a9c4-b1976986ef50 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:02.109019 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01659466-9792-4dce-a9c4-b1976986ef50 None admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:02.109789 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01659466-9792-4dce-a9c4-b1976986ef50 None admin] RBAC: Adding query filter params (name=member) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:58:02.110128 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01659466-9792-4dce-a9c4-b1976986ef50 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:02.111613 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01659466-9792-4dce-a9c4-b1976986ef50 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:02.123069 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 68/130] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Sep 6 19:58:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:02.124436 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ffcf7b-ac22-4762-9ce1-57b779511276 None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:02.124728 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ffcf7b-ac22-4762-9ce1-57b779511276 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:02.125010 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ffcf7b-ac22-4762-9ce1-57b779511276 None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:02.126996 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 69/131] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 19:58:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 19:58:02.127585 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ffcf7b-ac22-4762-9ce1-57b779511276 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:02.163484 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b472ffce-13bd-4d7b-9a4d-ee4ab33c55d1 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:02.223968 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab091fd-d80f-4505-a86c-2e34f00524a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:02.224699 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab091fd-d80f-4505-a86c-2e34f00524a7 None admin] REQUEST_METHOD: `PUT` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:02.224990 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab091fd-d80f-4505-a86c-2e34f00524a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:02.225310 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab091fd-d80f-4505-a86c-2e34f00524a7 None admin] PATH_INFO: `/v3/projects/197e55aa210c42abbf12e585a968f8c5/users/1e7095a0137b45c984179dcf25de9362/roles/e0ab41d8ae22474688b5361bcc75e0f0` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:02.227990 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab091fd-d80f-4505-a86c-2e34f00524a7 None admin] RBAC: Authorizing `identity:create_grant(project_id=197e55aa210c42abbf12e585a968f8c5, user_id=1e7095a0137b45c984179dcf25de9362, role_id=e0ab41d8ae22474688b5361bcc75e0f0)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:02.230222 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab091fd-d80f-4505-a86c-2e34f00524a7 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:02.247926 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 63/132] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Sep 6 19:58:01 2022] PUT /identity/v3/projects/197e55aa210c42abbf12e585a968f8c5/users/1e7095a0137b45c984179dcf25de9362/roles/e0ab41d8ae22474688b5361bcc75e0f0 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 19:58:02.248792 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab091fd-d80f-4505-a86c-2e34f00524a7 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:02.342919 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ffcf7b-ac22-4762-9ce1-57b779511276 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:02.372154 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:02.392508 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9742a654-5acc-4f4b-a32e-f527797550ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:02.393484 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9742a654-5acc-4f4b-a32e-f527797550ab None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:02.393802 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9742a654-5acc-4f4b-a32e-f527797550ab None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:02.394077 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9742a654-5acc-4f4b-a32e-f527797550ab None admin] PATH_INFO: `/v3/projects/197e55aa210c42abbf12e585a968f8c5` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:02.395442 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9742a654-5acc-4f4b-a32e-f527797550ab None admin] RBAC: Authorizing `identity:get_project(project_id=197e55aa210c42abbf12e585a968f8c5)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:02.397077 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9742a654-5acc-4f4b-a32e-f527797550ab None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:02.399958 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 70/133] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 6 19:58:02 2022] GET /identity/v3/projects/197e55aa210c42abbf12e585a968f8c5 => generated 298 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:02.400907 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9742a654-5acc-4f4b-a32e-f527797550ab None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:02.459867 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab091fd-d80f-4505-a86c-2e34f00524a7 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:02.477078 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:02.505545 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888571a9-62b7-4752-9be2-64dc4a9bf1e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:02.507557 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888571a9-62b7-4752-9be2-64dc4a9bf1e4 None admin] REQUEST_METHOD: `PUT` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:02.507557 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888571a9-62b7-4752-9be2-64dc4a9bf1e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:02.507771 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888571a9-62b7-4752-9be2-64dc4a9bf1e4 None admin] PATH_INFO: `/v3/projects/197e55aa210c42abbf12e585a968f8c5/users/a729c9f9f70c41eab4ccf79f7ed5ff84/roles/b02170460f054c259e825ab6e35f8537` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:02.512132 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888571a9-62b7-4752-9be2-64dc4a9bf1e4 None admin] RBAC: Authorizing `identity:create_grant(project_id=197e55aa210c42abbf12e585a968f8c5, user_id=a729c9f9f70c41eab4ccf79f7ed5ff84, role_id=b02170460f054c259e825ab6e35f8537)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:02.514708 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888571a9-62b7-4752-9be2-64dc4a9bf1e4 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:02.538899 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 64/134] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Sep 6 19:58:02 2022] PUT /identity/v3/projects/197e55aa210c42abbf12e585a968f8c5/users/a729c9f9f70c41eab4ccf79f7ed5ff84/roles/b02170460f054c259e825ab6e35f8537 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 19:58:02.541351 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0b01d2-d129-4ca3-8943-90e93d212399 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:02.541901 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0b01d2-d129-4ca3-8943-90e93d212399 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:02.542475 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0b01d2-d129-4ca3-8943-90e93d212399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:02.568581 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-0d0b01d2-d129-4ca3-8943-90e93d212399 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 19:58:02.570239 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-0d0b01d2-d129-4ca3-8943-90e93d212399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 19:58:02.602602 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9742a654-5acc-4f4b-a32e-f527797550ab None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:02.671970 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 65/135] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 19:58:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 19:58:02.673148 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0b01d2-d129-4ca3-8943-90e93d212399 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:02.704029 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015e9aff-51e1-4cf7-bd7d-8494bd15fada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:02.705155 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015e9aff-51e1-4cf7-bd7d-8494bd15fada None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:02.705708 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015e9aff-51e1-4cf7-bd7d-8494bd15fada None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:02.706209 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015e9aff-51e1-4cf7-bd7d-8494bd15fada None admin] PATH_INFO: `/v3/users/a729c9f9f70c41eab4ccf79f7ed5ff84` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:02.710651 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015e9aff-51e1-4cf7-bd7d-8494bd15fada None admin] RBAC: Authorizing `identity:get_user(user_id=a729c9f9f70c41eab4ccf79f7ed5ff84)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:02.712811 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015e9aff-51e1-4cf7-bd7d-8494bd15fada None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:02.719145 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 71/136] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 6 19:58:02 2022] GET /identity/v3/users/a729c9f9f70c41eab4ccf79f7ed5ff84 => generated 275 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:02.726337 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015e9aff-51e1-4cf7-bd7d-8494bd15fada None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:02.782040 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:02.831366 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0b01d2-d129-4ca3-8943-90e93d212399 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:02.880804 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da170aac-609c-4fdf-9ece-b2e9a8655ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:02.881440 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da170aac-609c-4fdf-9ece-b2e9a8655ac8 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:02.881770 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da170aac-609c-4fdf-9ece-b2e9a8655ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:02.882052 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da170aac-609c-4fdf-9ece-b2e9a8655ac8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:02.883447 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da170aac-609c-4fdf-9ece-b2e9a8655ac8 None admin] RBAC: Adding query filter params (role.id=034691b44a684a7b8b9bf9f601dd64e9, scope.project.id=197e55aa210c42abbf12e585a968f8c5, user.id=37e092fe6c824b3cb149b714145f1710) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:58:02.883769 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da170aac-609c-4fdf-9ece-b2e9a8655ac8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:02.885489 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da170aac-609c-4fdf-9ece-b2e9a8655ac8 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:02.896073 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 66/137] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Sep 6 19:58:02 2022] GET /identity/v3/role_assignments?user.id=37e092fe6c824b3cb149b714145f1710&scope.project.id=197e55aa210c42abbf12e585a968f8c5&role.id=034691b44a684a7b8b9bf9f601dd64e9 => generated 259 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:02.926234 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:02.934596 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015e9aff-51e1-4cf7-bd7d-8494bd15fada None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:02.997858 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6793eb-c6b8-4120-bac2-7f30ebd7cdf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:02.998472 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6793eb-c6b8-4120-bac2-7f30ebd7cdf9 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:02.998758 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6793eb-c6b8-4120-bac2-7f30ebd7cdf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:02.999041 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6793eb-c6b8-4120-bac2-7f30ebd7cdf9 None admin] PATH_INFO: `/v3/projects/183ec5fbd9a04da1ab46d9eaccdf955a` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:03.002652 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6793eb-c6b8-4120-bac2-7f30ebd7cdf9 None admin] RBAC: Authorizing `identity:get_project(project_id=183ec5fbd9a04da1ab46d9eaccdf955a)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:03.004711 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6793eb-c6b8-4120-bac2-7f30ebd7cdf9 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:03.007114 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 72/138] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 6 19:58:02 2022] GET /identity/v3/projects/183ec5fbd9a04da1ab46d9eaccdf955a => generated 312 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 19:58:03.014318 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da170aac-609c-4fdf-9ece-b2e9a8655ac8 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:03.210353 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da170aac-609c-4fdf-9ece-b2e9a8655ac8 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:03.243894 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:03.266944 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760f34c5-5236-497c-a327-73a49e1a8070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:03.267822 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760f34c5-5236-497c-a327-73a49e1a8070 None admin] REQUEST_METHOD: `PUT` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:03.268110 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760f34c5-5236-497c-a327-73a49e1a8070 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:03.268401 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760f34c5-5236-497c-a327-73a49e1a8070 None admin] PATH_INFO: `/v3/projects/183ec5fbd9a04da1ab46d9eaccdf955a/users/a729c9f9f70c41eab4ccf79f7ed5ff84/roles/4eea39e24b06431aa357053fe7907a3d` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:03.275413 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760f34c5-5236-497c-a327-73a49e1a8070 None admin] RBAC: Authorizing `identity:create_grant(project_id=183ec5fbd9a04da1ab46d9eaccdf955a, user_id=a729c9f9f70c41eab4ccf79f7ed5ff84, role_id=4eea39e24b06431aa357053fe7907a3d)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:03.276993 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760f34c5-5236-497c-a327-73a49e1a8070 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:03.299717 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 67/139] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Sep 6 19:58:03 2022] PUT /identity/v3/projects/183ec5fbd9a04da1ab46d9eaccdf955a/users/a729c9f9f70c41eab4ccf79f7ed5ff84/roles/4eea39e24b06431aa357053fe7907a3d => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 19:58:03.386994 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6793eb-c6b8-4120-bac2-7f30ebd7cdf9 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:03.635065 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6793eb-c6b8-4120-bac2-7f30ebd7cdf9 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:03.722871 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75958dd6-7c42-47cd-9697-ed9cb577ce1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:03.723993 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75958dd6-7c42-47cd-9697-ed9cb577ce1b None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:03.724532 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75958dd6-7c42-47cd-9697-ed9cb577ce1b None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:03.725133 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75958dd6-7c42-47cd-9697-ed9cb577ce1b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:03.727976 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75958dd6-7c42-47cd-9697-ed9cb577ce1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:03.730973 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75958dd6-7c42-47cd-9697-ed9cb577ce1b None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:03.734444 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 73/140] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 19:58:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:03.740493 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760f34c5-5236-497c-a327-73a49e1a8070 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:03.911464 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760f34c5-5236-497c-a327-73a49e1a8070 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:03.970526 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4a5510-4e0e-45a5-8717-20e5a303c2a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:03.971378 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4a5510-4e0e-45a5-8717-20e5a303c2a6 None admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:03.971759 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4a5510-4e0e-45a5-8717-20e5a303c2a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:03.972043 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4a5510-4e0e-45a5-8717-20e5a303c2a6 None admin] PATH_INFO: `/v3/users` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:03.972777 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4a5510-4e0e-45a5-8717-20e5a303c2a6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:03.974637 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4a5510-4e0e-45a5-8717-20e5a303c2a6 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:04.009007 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 68/141] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Sep 6 19:58:03 2022] POST /identity/v3/users => generated 283 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 19:58:05.966171 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf50f6d-268c-416a-ac3f-8dc6dbeda330 None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:05.967786 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf50f6d-268c-416a-ac3f-8dc6dbeda330 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:05.968521 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf50f6d-268c-416a-ac3f-8dc6dbeda330 None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:05.971115 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 74/142] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 19:58:05 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 19:58:05.981566 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f02f0fb-b26a-439e-9e57-3dfa3e707bdc None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:05.983056 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f02f0fb-b26a-439e-9e57-3dfa3e707bdc None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:05.983802 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f02f0fb-b26a-439e-9e57-3dfa3e707bdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:06.023971 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-5f02f0fb-b26a-439e-9e57-3dfa3e707bdc None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 19:58:06.026413 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-5f02f0fb-b26a-439e-9e57-3dfa3e707bdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 19:58:06.088627 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:06.090175 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5b31fc-9b3a-4a8d-8ee3-9659f39317df None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:06.090680 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5b31fc-9b3a-4a8d-8ee3-9659f39317df None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:06.091168 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5b31fc-9b3a-4a8d-8ee3-9659f39317df None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:06.098263 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 75/143] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 19:58:06 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 19:58:06.109674 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3ba06b-e20c-4f82-986b-170ec0bf6849 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:06.110201 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3ba06b-e20c-4f82-986b-170ec0bf6849 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:06.110719 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3ba06b-e20c-4f82-986b-170ec0bf6849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:06.167742 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-7f3ba06b-e20c-4f82-986b-170ec0bf6849 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 19:58:06.170067 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-7f3ba06b-e20c-4f82-986b-170ec0bf6849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 19:58:06.180441 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 69/144] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 19:58:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 19:58:06.259385 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:06.355408 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 76/145] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 19:58:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 19:58:06.361515 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e8457a-6901-4075-bdb0-82030000fd0b None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:06.361925 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e8457a-6901-4075-bdb0-82030000fd0b None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:06.362214 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e8457a-6901-4075-bdb0-82030000fd0b None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:06.364036 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 70/146] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 19:58:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 19:58:06.371796 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05824bc0-8a54-45e1-90dc-baaa546b4af6 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:06.372292 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05824bc0-8a54-45e1-90dc-baaa546b4af6 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:06.372781 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05824bc0-8a54-45e1-90dc-baaa546b4af6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:06.406941 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-05824bc0-8a54-45e1-90dc-baaa546b4af6 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 19:58:06.408636 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-05824bc0-8a54-45e1-90dc-baaa546b4af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 19:58:06.520639 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 77/147] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 19:58:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 19:58:06.847354 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e369ac-9e0a-40a8-abc4-a6b7e6ac0a6d None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:06.847640 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e369ac-9e0a-40a8-abc4-a6b7e6ac0a6d None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:06.848119 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e369ac-9e0a-40a8-abc4-a6b7e6ac0a6d None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:06.850081 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 71/148] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 19:58:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 19:58:06.862525 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f85dd89-183a-43d9-ab8c-44be2ed016da None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:06.863053 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f85dd89-183a-43d9-ab8c-44be2ed016da None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:06.863500 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f85dd89-183a-43d9-ab8c-44be2ed016da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:06.889299 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-2f85dd89-183a-43d9-ab8c-44be2ed016da None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 19:58:06.890851 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-2f85dd89-183a-43d9-ab8c-44be2ed016da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 19:58:07.001190 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 78/149] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 19:58:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 19:58:07.085043 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e369ac-9e0a-40a8-abc4-a6b7e6ac0a6d None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:07.142637 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f85dd89-183a-43d9-ab8c-44be2ed016da None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:07.201260 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:07.347829 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e369ac-9e0a-40a8-abc4-a6b7e6ac0a6d None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:07.405575 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f85dd89-183a-43d9-ab8c-44be2ed016da None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:07.436026 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151fd721-aea4-451f-9b5d-41c21d7ed0d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:07.437509 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151fd721-aea4-451f-9b5d-41c21d7ed0d8 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:07.438016 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151fd721-aea4-451f-9b5d-41c21d7ed0d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:07.439125 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151fd721-aea4-451f-9b5d-41c21d7ed0d8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:07.476235 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151fd721-aea4-451f-9b5d-41c21d7ed0d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 19:58:07.494178 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-151fd721-aea4-451f-9b5d-41c21d7ed0d8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 19:58:07.494178 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:58:07.494178 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:58:07.494178 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:58:07.494178 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:58:07.494178 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:58:07.494178 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:58:07.494178 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:58:07.494178 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:58:07.494178 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:58:07.494178 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:58:07.494178 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:58:07.494178 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 19:58:07.494178 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 19:58:07.494178 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 19:58:07.494178 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 19:58:07.495259 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 19:58:07.495259 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 19:58:07.495259 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 19:58:07.495259 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 19:58:07.495259 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:58:07.495259 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:58:07.495259 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:58:07.495259 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:58:07.495259 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:58:07.495259 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:58:07.495259 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:58:07.495259 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 19:58:07.495259 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:58:07.495259 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:58:07.495259 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:58:07.495259 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:58:07.496328 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:58:07.496328 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:58:07.496328 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:58:07.496328 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:58:07.496328 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 19:58:07.496328 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 19:58:07.496328 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 19:58:07.496328 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 19:58:07.496328 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 19:58:07.496328 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 19:58:07.496328 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 19:58:07.496328 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 19:58:07.496958 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 72/150] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 6 19:58:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 413 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:58:07.498117 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151fd721-aea4-451f-9b5d-41c21d7ed0d8 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:07.536874 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f30b6d-5ef4-41c6-ae18-4a929d2e1f70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:07.538671 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f30b6d-5ef4-41c6-ae18-4a929d2e1f70 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:07.539180 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f30b6d-5ef4-41c6-ae18-4a929d2e1f70 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:07.539686 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f30b6d-5ef4-41c6-ae18-4a929d2e1f70 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:07.567415 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f30b6d-5ef4-41c6-ae18-4a929d2e1f70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 19:58:07.573284 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-67f30b6d-5ef4-41c6-ae18-4a929d2e1f70 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 19:58:07.573284 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:58:07.573284 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:58:07.573284 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:58:07.573284 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:58:07.573284 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:58:07.573284 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:58:07.573284 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:58:07.573284 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:58:07.573284 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:58:07.573284 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:58:07.573284 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:58:07.573284 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 19:58:07.573284 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 19:58:07.573284 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 19:58:07.573284 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 19:58:07.574651 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 19:58:07.574651 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 19:58:07.574651 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 19:58:07.574651 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 19:58:07.574651 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:58:07.574651 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:58:07.574651 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:58:07.574651 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:58:07.574651 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:58:07.574651 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:58:07.574651 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:58:07.574651 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 19:58:07.574651 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:58:07.574651 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:58:07.574651 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:58:07.574651 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:58:07.575444 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:58:07.575444 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:58:07.575444 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:58:07.575444 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:58:07.575444 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 19:58:07.575444 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 19:58:07.575444 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 19:58:07.575444 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 19:58:07.575444 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 19:58:07.575444 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 19:58:07.575444 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 19:58:07.575444 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 19:58:07.582138 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 79/151] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 6 19:58:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 440 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 19:58:07.583080 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:07.584404 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f30b6d-5ef4-41c6-ae18-4a929d2e1f70 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:07.785698 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151fd721-aea4-451f-9b5d-41c21d7ed0d8 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:07.880004 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2fac07-18bc-4be8-8226-8a4bea0a6ae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:07.881147 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2fac07-18bc-4be8-8226-8a4bea0a6ae4 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:07.881629 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2fac07-18bc-4be8-8226-8a4bea0a6ae4 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:07.882125 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2fac07-18bc-4be8-8226-8a4bea0a6ae4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:07.890340 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:07.924155 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2fac07-18bc-4be8-8226-8a4bea0a6ae4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 19:58:07.935068 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f30b6d-5ef4-41c6-ae18-4a929d2e1f70 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:07.943814 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-5b2fac07-18bc-4be8-8226-8a4bea0a6ae4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 19:58:07.943814 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:58:07.943814 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:58:07.943814 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:58:07.943814 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:58:07.943814 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:58:07.943814 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:58:07.943814 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:58:07.943814 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:58:07.943814 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:58:07.943814 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:58:07.943814 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:58:07.943814 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 19:58:07.943814 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 19:58:07.943814 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 19:58:07.943814 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 19:58:07.944747 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 19:58:07.944747 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 19:58:07.944747 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 19:58:07.944747 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 19:58:07.944747 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:58:07.944747 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:58:07.944747 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:58:07.944747 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:58:07.944747 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:58:07.944747 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:58:07.944747 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:58:07.944747 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 19:58:07.944747 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:58:07.944747 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:58:07.944747 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:58:07.944747 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:58:07.945549 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:58:07.945549 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:58:07.945549 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:58:07.945549 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:58:07.945549 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 19:58:07.945549 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 19:58:07.945549 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 19:58:07.945549 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 19:58:07.945549 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 19:58:07.945549 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 19:58:07.945549 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 19:58:07.945549 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 19:58:07.945549 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 73/152] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Sep 6 19:58:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 444 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:58:07.945549 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2fac07-18bc-4be8-8226-8a4bea0a6ae4 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:08.012649 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5d5717-62d7-4a4f-a2e7-c8f4925cd0d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:08.016080 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5d5717-62d7-4a4f-a2e7-c8f4925cd0d4 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:08.017289 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5d5717-62d7-4a4f-a2e7-c8f4925cd0d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:08.018080 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5d5717-62d7-4a4f-a2e7-c8f4925cd0d4 None admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:08.020254 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5d5717-62d7-4a4f-a2e7-c8f4925cd0d4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:58:08.021779 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5d5717-62d7-4a4f-a2e7-c8f4925cd0d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:08.025196 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5d5717-62d7-4a4f-a2e7-c8f4925cd0d4 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:08.055818 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 80/153] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Sep 6 19:58:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 473 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:08.057194 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5d5717-62d7-4a4f-a2e7-c8f4925cd0d4 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:08.184312 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:08.222789 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2fac07-18bc-4be8-8226-8a4bea0a6ae4 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:08.287612 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5d5717-62d7-4a4f-a2e7-c8f4925cd0d4 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:08.339778 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ec8b04-4b70-4486-97ea-8110c4fe72f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:08.340841 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ec8b04-4b70-4486-97ea-8110c4fe72f6 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:08.341374 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ec8b04-4b70-4486-97ea-8110c4fe72f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:08.345918 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ec8b04-4b70-4486-97ea-8110c4fe72f6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:08.363318 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ec8b04-4b70-4486-97ea-8110c4fe72f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 19:58:08.379426 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-f3ec8b04-4b70-4486-97ea-8110c4fe72f6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 19:58:08.379426 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:58:08.379426 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:58:08.379426 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:58:08.379426 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:58:08.379426 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:58:08.379426 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:58:08.379426 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:58:08.379426 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:58:08.379426 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:58:08.379426 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:58:08.379426 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:58:08.379426 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 19:58:08.379426 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 19:58:08.379426 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 19:58:08.379426 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 19:58:08.380491 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 19:58:08.380491 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 19:58:08.380491 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 19:58:08.380491 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 19:58:08.380491 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:58:08.380491 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:58:08.380491 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:58:08.380491 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:58:08.380491 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:58:08.380491 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:58:08.380491 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:58:08.380491 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 19:58:08.380491 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:58:08.380491 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:58:08.380491 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:58:08.380491 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:58:08.381265 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:58:08.381265 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:58:08.381265 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:58:08.381265 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:58:08.381265 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 19:58:08.381265 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 19:58:08.381265 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 19:58:08.381265 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 19:58:08.381265 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 19:58:08.381265 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 19:58:08.381265 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 19:58:08.381265 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 19:58:08.383956 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 81/154] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Sep 6 19:58:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:58:08.385045 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ec8b04-4b70-4486-97ea-8110c4fe72f6 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:08.385472 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243186ec-9f91-480e-b565-807d674e9417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:08.389222 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243186ec-9f91-480e-b565-807d674e9417 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:08.389222 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243186ec-9f91-480e-b565-807d674e9417 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:08.389222 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243186ec-9f91-480e-b565-807d674e9417 None admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:08.389222 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243186ec-9f91-480e-b565-807d674e9417 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:58:08.389866 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243186ec-9f91-480e-b565-807d674e9417 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:08.392777 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243186ec-9f91-480e-b565-807d674e9417 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:08.413650 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:08.441842 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 74/155] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Sep 6 19:58:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 499 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:08.443466 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243186ec-9f91-480e-b565-807d674e9417 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:08.568060 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ec8b04-4b70-4486-97ea-8110c4fe72f6 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:08.634015 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286806da-c9bc-400c-ade4-bb5eb6eec846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:08.634646 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286806da-c9bc-400c-ade4-bb5eb6eec846 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:08.634932 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286806da-c9bc-400c-ade4-bb5eb6eec846 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:08.635208 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286806da-c9bc-400c-ade4-bb5eb6eec846 None admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:08.635920 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286806da-c9bc-400c-ade4-bb5eb6eec846 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:58:08.636276 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286806da-c9bc-400c-ade4-bb5eb6eec846 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:08.638242 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286806da-c9bc-400c-ade4-bb5eb6eec846 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:08.654474 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 82/156] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 6 19:58:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:08.655466 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286806da-c9bc-400c-ade4-bb5eb6eec846 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:08.787568 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:08.823240 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243186ec-9f91-480e-b565-807d674e9417 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:08.839939 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286806da-c9bc-400c-ade4-bb5eb6eec846 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:08.870775 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0854fc10-586f-413d-95f9-6ba8c218ee19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:08.871797 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0854fc10-586f-413d-95f9-6ba8c218ee19 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:08.872353 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0854fc10-586f-413d-95f9-6ba8c218ee19 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:08.872944 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0854fc10-586f-413d-95f9-6ba8c218ee19 None admin] PATH_INFO: `/v3/users/a729c9f9f70c41eab4ccf79f7ed5ff84` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:08.874671 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0854fc10-586f-413d-95f9-6ba8c218ee19 None admin] RBAC: Authorizing `identity:get_user(user_id=a729c9f9f70c41eab4ccf79f7ed5ff84)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:08.876426 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0854fc10-586f-413d-95f9-6ba8c218ee19 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:08.878619 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 75/157] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 6 19:58:08 2022] GET /identity/v3/users/a729c9f9f70c41eab4ccf79f7ed5ff84 => generated 275 bytes in 436 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:08.879766 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0854fc10-586f-413d-95f9-6ba8c218ee19 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:08.900513 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:08.918625 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e37a0a-554c-467d-9b82-9f1573bb227f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:08.919621 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e37a0a-554c-467d-9b82-9f1573bb227f None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:08.920105 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e37a0a-554c-467d-9b82-9f1573bb227f None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:08.920777 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e37a0a-554c-467d-9b82-9f1573bb227f None admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:08.921743 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e37a0a-554c-467d-9b82-9f1573bb227f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:58:08.922273 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e37a0a-554c-467d-9b82-9f1573bb227f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:08.925338 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e37a0a-554c-467d-9b82-9f1573bb227f None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:08.942402 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 83/158] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 6 19:58:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:08.944314 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e37a0a-554c-467d-9b82-9f1573bb227f None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:09.112785 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e37a0a-554c-467d-9b82-9f1573bb227f None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:09.155808 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0854fc10-586f-413d-95f9-6ba8c218ee19 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:09.166211 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:09.178203 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291de9c2-7d37-4edc-8bff-d0a3e3871914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:09.178994 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291de9c2-7d37-4edc-8bff-d0a3e3871914 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:09.179282 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291de9c2-7d37-4edc-8bff-d0a3e3871914 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:09.179586 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291de9c2-7d37-4edc-8bff-d0a3e3871914 None admin] PATH_INFO: `/v3/users/1e7095a0137b45c984179dcf25de9362` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:09.181123 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291de9c2-7d37-4edc-8bff-d0a3e3871914 None admin] RBAC: Authorizing `identity:get_user(user_id=1e7095a0137b45c984179dcf25de9362)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:09.182747 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291de9c2-7d37-4edc-8bff-d0a3e3871914 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:09.184451 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 84/159] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 6 19:58:08 2022] GET /identity/v3/users/1e7095a0137b45c984179dcf25de9362 => generated 247 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:09.191519 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4865bf8-cbc9-4256-842c-a14e099e151f None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:09.191807 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4865bf8-cbc9-4256-842c-a14e099e151f None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:09.192113 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4865bf8-cbc9-4256-842c-a14e099e151f None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:09.198467 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 85/160] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 19:58:09 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 19:58:09.199612 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4865bf8-cbc9-4256-842c-a14e099e151f None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:09.241650 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b570687-16c3-4c85-a6fd-fce2ea3b4197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:09.242421 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b570687-16c3-4c85-a6fd-fce2ea3b4197 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:09.242739 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b570687-16c3-4c85-a6fd-fce2ea3b4197 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:09.242970 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b570687-16c3-4c85-a6fd-fce2ea3b4197 None admin] PATH_INFO: `/v3/users/37e092fe6c824b3cb149b714145f1710` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:09.248590 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b570687-16c3-4c85-a6fd-fce2ea3b4197 None admin] RBAC: Authorizing `identity:get_user(user_id=37e092fe6c824b3cb149b714145f1710)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:09.250294 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b570687-16c3-4c85-a6fd-fce2ea3b4197 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:09.255902 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 76/161] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 6 19:58:08 2022] GET /identity/v3/users/37e092fe6c824b3cb149b714145f1710 => generated 289 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:09.257199 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b570687-16c3-4c85-a6fd-fce2ea3b4197 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:09.411085 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:09.437395 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b570687-16c3-4c85-a6fd-fce2ea3b4197 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:09.499181 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a504ab8-1ef2-4be6-9066-336c02b5fb4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:09.500144 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a504ab8-1ef2-4be6-9066-336c02b5fb4c None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:09.500701 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a504ab8-1ef2-4be6-9066-336c02b5fb4c None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:09.501159 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a504ab8-1ef2-4be6-9066-336c02b5fb4c None admin] PATH_INFO: `/v3/users/a729c9f9f70c41eab4ccf79f7ed5ff84` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:09.507016 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a504ab8-1ef2-4be6-9066-336c02b5fb4c None admin] RBAC: Authorizing `identity:get_user(user_id=a729c9f9f70c41eab4ccf79f7ed5ff84)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:09.510254 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a504ab8-1ef2-4be6-9066-336c02b5fb4c None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:09.512580 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 77/162] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 6 19:58:09 2022] GET /identity/v3/users/a729c9f9f70c41eab4ccf79f7ed5ff84 => generated 275 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:09.517670 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a504ab8-1ef2-4be6-9066-336c02b5fb4c None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:09.518568 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4865bf8-cbc9-4256-842c-a14e099e151f None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:09.661926 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23c785c-85ba-4d9b-9f7c-338889229b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:09.666475 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23c785c-85ba-4d9b-9f7c-338889229b7a None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:09.674086 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23c785c-85ba-4d9b-9f7c-338889229b7a None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:09.675325 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23c785c-85ba-4d9b-9f7c-338889229b7a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:09.790709 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a504ab8-1ef2-4be6-9066-336c02b5fb4c None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:09.794916 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:09.799481 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23c785c-85ba-4d9b-9f7c-338889229b7a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:09.801979 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23c785c-85ba-4d9b-9f7c-338889229b7a None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:09.857531 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08a36da-efa6-4bab-a633-da823dffedbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:09.858623 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08a36da-efa6-4bab-a633-da823dffedbf None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:09.859122 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08a36da-efa6-4bab-a633-da823dffedbf None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:09.859629 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08a36da-efa6-4bab-a633-da823dffedbf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:09.896742 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:09.943367 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 86/163] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 6 19:58:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 739 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 19:58:09.943367 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e36656-6e54-4fbb-b4e9-3c0ef4b77d33 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:09.943367 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e36656-6e54-4fbb-b4e9-3c0ef4b77d33 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:09.943367 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e36656-6e54-4fbb-b4e9-3c0ef4b77d33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:09.962290 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-b1e36656-6e54-4fbb-b4e9-3c0ef4b77d33 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 19:58:09.963683 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-b1e36656-6e54-4fbb-b4e9-3c0ef4b77d33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 19:58:10.003409 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08a36da-efa6-4bab-a633-da823dffedbf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:10.006534 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08a36da-efa6-4bab-a633-da823dffedbf None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:10.081685 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 87/164] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 19:58:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 19:58:10.085819 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e36656-6e54-4fbb-b4e9-3c0ef4b77d33 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:10.126961 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:10.192167 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 78/165] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 6 19:58:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 679 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 19:58:10.193836 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08a36da-efa6-4bab-a633-da823dffedbf None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:10.226928 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e36656-6e54-4fbb-b4e9-3c0ef4b77d33 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:10.271407 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27f9f2e-98b5-45f5-a37e-f5414dce1e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:10.273734 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27f9f2e-98b5-45f5-a37e-f5414dce1e49 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:10.274302 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27f9f2e-98b5-45f5-a37e-f5414dce1e49 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:10.274782 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27f9f2e-98b5-45f5-a37e-f5414dce1e49 None admin] PATH_INFO: `/v3/projects/197e55aa210c42abbf12e585a968f8c5` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:10.276380 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27f9f2e-98b5-45f5-a37e-f5414dce1e49 None admin] RBAC: Authorizing `identity:get_project(project_id=197e55aa210c42abbf12e585a968f8c5)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:10.276904 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:10.279099 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27f9f2e-98b5-45f5-a37e-f5414dce1e49 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:10.281495 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 88/166] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 6 19:58:10 2022] GET /identity/v3/projects/197e55aa210c42abbf12e585a968f8c5 => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:10.283527 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65014dfc-1e22-4cfe-8924-73639f9e26bb None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:10.284117 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65014dfc-1e22-4cfe-8924-73639f9e26bb None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:10.284592 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65014dfc-1e22-4cfe-8924-73639f9e26bb None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:10.286760 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 89/167] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 19:58:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 19:58:10.288585 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5f4301-8549-4c06-8695-e8a420918005 None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:10.289149 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5f4301-8549-4c06-8695-e8a420918005 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:10.290089 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5f4301-8549-4c06-8695-e8a420918005 None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:10.291739 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 90/168] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 19:58:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 19:58:10.292838 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5f4301-8549-4c06-8695-e8a420918005 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:10.391835 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08a36da-efa6-4bab-a633-da823dffedbf None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:10.410772 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:10.424126 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5f4301-8549-4c06-8695-e8a420918005 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:10.441238 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eab114-55b8-403c-8b61-275a3cc3a2d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:10.441907 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eab114-55b8-403c-8b61-275a3cc3a2d2 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:10.442803 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eab114-55b8-403c-8b61-275a3cc3a2d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:10.443090 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eab114-55b8-403c-8b61-275a3cc3a2d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:10.488647 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92651b2d-5aba-48c8-b1b4-86b86ecfc7a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:10.490453 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92651b2d-5aba-48c8-b1b4-86b86ecfc7a4 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:10.490854 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92651b2d-5aba-48c8-b1b4-86b86ecfc7a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:10.491247 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92651b2d-5aba-48c8-b1b4-86b86ecfc7a4 None admin] PATH_INFO: `/v3/projects/197e55aa210c42abbf12e585a968f8c5` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:10.493190 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92651b2d-5aba-48c8-b1b4-86b86ecfc7a4 None admin] RBAC: Authorizing `identity:get_project(project_id=197e55aa210c42abbf12e585a968f8c5)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:10.495959 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92651b2d-5aba-48c8-b1b4-86b86ecfc7a4 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:10.504089 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 91/169] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 6 19:58:10 2022] GET /identity/v3/projects/197e55aa210c42abbf12e585a968f8c5 => generated 298 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:10.504089 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92651b2d-5aba-48c8-b1b4-86b86ecfc7a4 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:10.546702 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3eab114-55b8-403c-8b61-275a3cc3a2d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:10.549680 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3eab114-55b8-403c-8b61-275a3cc3a2d2 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:10.621886 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:10.638126 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92651b2d-5aba-48c8-b1b4-86b86ecfc7a4 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:10.689804 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 79/170] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 6 19:58:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 19:58:10.689804 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eab114-55b8-403c-8b61-275a3cc3a2d2 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:10.690059 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d9392f-487e-4dc5-96b6-f6a1e518c52f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:10.690059 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d9392f-487e-4dc5-96b6-f6a1e518c52f None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:10.690059 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d9392f-487e-4dc5-96b6-f6a1e518c52f None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:10.690059 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d9392f-487e-4dc5-96b6-f6a1e518c52f None admin] PATH_INFO: `/v3/projects/197e55aa210c42abbf12e585a968f8c5` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:10.690366 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d9392f-487e-4dc5-96b6-f6a1e518c52f None admin] RBAC: Authorizing `identity:get_project(project_id=197e55aa210c42abbf12e585a968f8c5)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:10.691479 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d9392f-487e-4dc5-96b6-f6a1e518c52f None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:10.693358 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 92/171] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 6 19:58:10 2022] GET /identity/v3/projects/197e55aa210c42abbf12e585a968f8c5 => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:10.694760 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b398696c-c67f-4603-97a6-c1cb153ff4c1 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:10.695071 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b398696c-c67f-4603-97a6-c1cb153ff4c1 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:10.695354 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b398696c-c67f-4603-97a6-c1cb153ff4c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:10.715939 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-b398696c-c67f-4603-97a6-c1cb153ff4c1 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 19:58:10.717184 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-b398696c-c67f-4603-97a6-c1cb153ff4c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 19:58:10.746738 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:10.797848 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 93/172] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 19:58:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 19:58:10.799369 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dbf742-a39b-412a-87d2-aa02a303a74b None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:10.799825 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dbf742-a39b-412a-87d2-aa02a303a74b None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:10.800127 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dbf742-a39b-412a-87d2-aa02a303a74b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:10.825244 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eab114-55b8-403c-8b61-275a3cc3a2d2 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:10.827197 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-13dbf742-a39b-412a-87d2-aa02a303a74b None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 19:58:10.829103 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-13dbf742-a39b-412a-87d2-aa02a303a74b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 19:58:10.891482 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375df04b-0b79-4003-98ec-b880ab22ec0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:10.892614 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375df04b-0b79-4003-98ec-b880ab22ec0f None admin] REQUEST_METHOD: `PUT` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:10.893083 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375df04b-0b79-4003-98ec-b880ab22ec0f None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:10.893512 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375df04b-0b79-4003-98ec-b880ab22ec0f None admin] PATH_INFO: `/v3/projects/197e55aa210c42abbf12e585a968f8c5/users/37e092fe6c824b3cb149b714145f1710/roles/034691b44a684a7b8b9bf9f601dd64e9` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:10.897476 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375df04b-0b79-4003-98ec-b880ab22ec0f None admin] RBAC: Authorizing `identity:create_grant(project_id=197e55aa210c42abbf12e585a968f8c5, user_id=37e092fe6c824b3cb149b714145f1710, role_id=034691b44a684a7b8b9bf9f601dd64e9)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:10.900896 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375df04b-0b79-4003-98ec-b880ab22ec0f None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:10.925372 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 80/173] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Sep 6 19:58:10 2022] PUT /identity/v3/projects/197e55aa210c42abbf12e585a968f8c5/users/37e092fe6c824b3cb149b714145f1710/roles/034691b44a684a7b8b9bf9f601dd64e9 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 19:58:10.927258 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f291482c-2fb8-4d13-be8d-caf9ba351662 None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:10.927779 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f291482c-2fb8-4d13-be8d-caf9ba351662 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:10.928266 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f291482c-2fb8-4d13-be8d-caf9ba351662 None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:10.930081 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 81/174] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 19:58:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 19:58:10.931248 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f291482c-2fb8-4d13-be8d-caf9ba351662 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:10.955891 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:10.974487 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 94/175] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 19:58:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 19:58:10.977190 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dbf742-a39b-412a-87d2-aa02a303a74b None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:11.153587 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f291482c-2fb8-4d13-be8d-caf9ba351662 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:11.184337 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:11.201936 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dbf742-a39b-412a-87d2-aa02a303a74b None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:11.236199 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e0b622-5d2b-48d8-8bba-6e9b1f57686a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:11.237385 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e0b622-5d2b-48d8-8bba-6e9b1f57686a None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:11.237930 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e0b622-5d2b-48d8-8bba-6e9b1f57686a None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:11.238433 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e0b622-5d2b-48d8-8bba-6e9b1f57686a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:11.240133 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e0b622-5d2b-48d8-8bba-6e9b1f57686a None admin] RBAC: Adding query filter params (role.id=4eea39e24b06431aa357053fe7907a3d, scope.project.id=197e55aa210c42abbf12e585a968f8c5, user.id=a729c9f9f70c41eab4ccf79f7ed5ff84) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:58:11.240742 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e0b622-5d2b-48d8-8bba-6e9b1f57686a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:11.243689 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e0b622-5d2b-48d8-8bba-6e9b1f57686a None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:11.258351 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 82/176] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Sep 6 19:58:10 2022] GET /identity/v3/role_assignments?user.id=a729c9f9f70c41eab4ccf79f7ed5ff84&scope.project.id=197e55aa210c42abbf12e585a968f8c5&role.id=4eea39e24b06431aa357053fe7907a3d => generated 608 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:11.259842 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e0b622-5d2b-48d8-8bba-6e9b1f57686a None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:11.270759 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f71cc72-5cee-4ea2-b46f-433dfd3d88f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:11.271813 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f71cc72-5cee-4ea2-b46f-433dfd3d88f9 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:11.272276 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f71cc72-5cee-4ea2-b46f-433dfd3d88f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:11.272725 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f71cc72-5cee-4ea2-b46f-433dfd3d88f9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:11.299116 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f71cc72-5cee-4ea2-b46f-433dfd3d88f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 19:58:11.309929 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-8f71cc72-5cee-4ea2-b46f-433dfd3d88f9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 19:58:11.309929 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:58:11.309929 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:58:11.309929 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:58:11.309929 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:58:11.309929 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:58:11.309929 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:58:11.309929 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:58:11.309929 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:58:11.309929 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:58:11.309929 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:58:11.309929 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:58:11.309929 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 19:58:11.309929 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 19:58:11.309929 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 19:58:11.309929 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 19:58:11.310955 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 19:58:11.310955 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 19:58:11.310955 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 19:58:11.310955 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 19:58:11.310955 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:58:11.310955 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:58:11.310955 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:58:11.310955 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:58:11.310955 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:58:11.310955 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:58:11.310955 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:58:11.310955 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 19:58:11.310955 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:58:11.310955 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:58:11.310955 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:58:11.310955 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:58:11.311952 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:58:11.311952 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:58:11.311952 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:58:11.311952 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:58:11.311952 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 19:58:11.311952 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 19:58:11.311952 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 19:58:11.311952 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 19:58:11.311952 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 19:58:11.311952 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 19:58:11.311952 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 19:58:11.311952 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 19:58:11.313768 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 95/177] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 6 19:58:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:58:11.313768 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f71cc72-5cee-4ea2-b46f-433dfd3d88f9 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:11.455863 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:11.525750 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e0b622-5d2b-48d8-8bba-6e9b1f57686a None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:11.599515 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee40c8b-eb79-422f-9ff4-549397ea038b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:11.600447 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f71cc72-5cee-4ea2-b46f-433dfd3d88f9 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:11.600802 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee40c8b-eb79-422f-9ff4-549397ea038b None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:11.601287 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee40c8b-eb79-422f-9ff4-549397ea038b None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:11.601796 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee40c8b-eb79-422f-9ff4-549397ea038b None admin] PATH_INFO: `/v3/projects/197e55aa210c42abbf12e585a968f8c5` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:11.603493 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee40c8b-eb79-422f-9ff4-549397ea038b None admin] RBAC: Authorizing `identity:get_project(project_id=197e55aa210c42abbf12e585a968f8c5)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:11.605199 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee40c8b-eb79-422f-9ff4-549397ea038b None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:11.607645 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 83/178] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 6 19:58:11 2022] GET /identity/v3/projects/197e55aa210c42abbf12e585a968f8c5 => generated 298 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:11.608971 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212b1a77-86d9-49a2-a37c-cd7636965747 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:11.609259 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212b1a77-86d9-49a2-a37c-cd7636965747 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:11.609691 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212b1a77-86d9-49a2-a37c-cd7636965747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:11.633196 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-212b1a77-86d9-49a2-a37c-cd7636965747 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 19:58:11.634536 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-212b1a77-86d9-49a2-a37c-cd7636965747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 19:58:11.695613 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9df00c-551a-424f-8d55-3a04e543948b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:11.696829 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9df00c-551a-424f-8d55-3a04e543948b None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:11.697504 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9df00c-551a-424f-8d55-3a04e543948b None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:11.698059 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9df00c-551a-424f-8d55-3a04e543948b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:11.699833 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9df00c-551a-424f-8d55-3a04e543948b None admin] RBAC: Adding query filter params (role.id=e0ab41d8ae22474688b5361bcc75e0f0, scope.project.id=197e55aa210c42abbf12e585a968f8c5, user.id=1e7095a0137b45c984179dcf25de9362) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:58:11.701814 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9df00c-551a-424f-8d55-3a04e543948b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:11.704672 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9df00c-551a-424f-8d55-3a04e543948b None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:11.716650 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 96/179] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Sep 6 19:58:11 2022] GET /identity/v3/role_assignments?user.id=1e7095a0137b45c984179dcf25de9362&scope.project.id=197e55aa210c42abbf12e585a968f8c5&role.id=e0ab41d8ae22474688b5361bcc75e0f0 => generated 608 bytes in 404 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:11.717483 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9df00c-551a-424f-8d55-3a04e543948b None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:11.767352 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 84/180] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 19:58:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 19:58:11.767848 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:11.768690 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212b1a77-86d9-49a2-a37c-cd7636965747 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:11.919710 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:11.924890 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9df00c-551a-424f-8d55-3a04e543948b None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:11.971805 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5c2b13-8ca3-4768-a4d4-ed7b55a3f74e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:11.973870 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5c2b13-8ca3-4768-a4d4-ed7b55a3f74e None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:11.974524 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5c2b13-8ca3-4768-a4d4-ed7b55a3f74e None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:11.975083 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5c2b13-8ca3-4768-a4d4-ed7b55a3f74e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:11.988794 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212b1a77-86d9-49a2-a37c-cd7636965747 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:11.990286 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5c2b13-8ca3-4768-a4d4-ed7b55a3f74e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 19:58:11.996590 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-ec5c2b13-8ca3-4768-a4d4-ed7b55a3f74e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 19:58:11.996590 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:58:11.996590 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:58:11.996590 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:58:11.996590 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:58:11.996590 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:58:11.996590 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:58:11.996590 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:58:11.996590 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:58:11.996590 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:58:11.996590 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:58:11.996590 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:58:11.996590 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 19:58:11.996590 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 19:58:11.996590 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 19:58:11.996590 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 19:58:11.997678 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 19:58:11.997678 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 19:58:11.997678 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 19:58:11.997678 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 19:58:11.997678 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:58:11.997678 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:58:11.997678 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:58:11.997678 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:58:11.997678 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:58:11.997678 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:58:11.997678 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:58:11.997678 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 19:58:11.997678 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:58:11.997678 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:58:11.997678 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:58:11.997678 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:58:11.998512 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:58:11.998512 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:58:11.998512 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:58:11.998512 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:58:11.998512 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 19:58:11.998512 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 19:58:11.998512 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 19:58:11.998512 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 19:58:11.998512 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 19:58:11.998512 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 19:58:11.998512 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 19:58:11.998512 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 19:58:12.000943 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 97/181] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Sep 6 19:58:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:58:12.002134 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5c2b13-8ca3-4768-a4d4-ed7b55a3f74e None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:12.052364 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b18ce6-8d0b-4f4c-9e05-65231ef800a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:12.053043 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b18ce6-8d0b-4f4c-9e05-65231ef800a3 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:12.053331 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b18ce6-8d0b-4f4c-9e05-65231ef800a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:12.053626 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b18ce6-8d0b-4f4c-9e05-65231ef800a3 None admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:12.054403 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b18ce6-8d0b-4f4c-9e05-65231ef800a3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:58:12.054766 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b18ce6-8d0b-4f4c-9e05-65231ef800a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:12.056582 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b18ce6-8d0b-4f4c-9e05-65231ef800a3 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:12.067616 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 85/182] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Sep 6 19:58:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:12.068852 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b18ce6-8d0b-4f4c-9e05-65231ef800a3 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:12.117309 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:12.153152 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5c2b13-8ca3-4768-a4d4-ed7b55a3f74e None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:12.196665 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baac3a7e-c329-45c4-81cc-c33ad69ac062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:12.198247 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baac3a7e-c329-45c4-81cc-c33ad69ac062 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:12.198758 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baac3a7e-c329-45c4-81cc-c33ad69ac062 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:12.199231 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baac3a7e-c329-45c4-81cc-c33ad69ac062 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:12.213168 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baac3a7e-c329-45c4-81cc-c33ad69ac062 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 19:58:12.219530 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-baac3a7e-c329-45c4-81cc-c33ad69ac062 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 19:58:12.219530 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:58:12.219530 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:58:12.219530 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:58:12.219530 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:58:12.219530 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:58:12.219530 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:58:12.219530 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:58:12.219530 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:58:12.219530 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:58:12.219530 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:58:12.219530 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:58:12.219530 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 19:58:12.219530 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 19:58:12.219530 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 19:58:12.219530 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 19:58:12.220665 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 19:58:12.220665 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 19:58:12.220665 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 19:58:12.220665 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 19:58:12.220665 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:58:12.220665 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:58:12.220665 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:58:12.220665 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:58:12.220665 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:58:12.220665 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:58:12.220665 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:58:12.220665 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 19:58:12.220665 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:58:12.220665 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:58:12.220665 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:58:12.220665 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:58:12.221477 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:58:12.221477 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:58:12.221477 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:58:12.221477 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:58:12.221477 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 19:58:12.221477 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 19:58:12.221477 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 19:58:12.221477 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 19:58:12.221477 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 19:58:12.221477 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 19:58:12.221477 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 19:58:12.221477 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 19:58:12.224209 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 98/183] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Sep 6 19:58:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:58:12.225365 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baac3a7e-c329-45c4-81cc-c33ad69ac062 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:12.279945 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b18ce6-8d0b-4f4c-9e05-65231ef800a3 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:12.327964 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfac80c-8d93-4e68-99dc-6e545ddf4a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:12.329017 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfac80c-8d93-4e68-99dc-6e545ddf4a28 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:12.329649 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfac80c-8d93-4e68-99dc-6e545ddf4a28 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:12.330148 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfac80c-8d93-4e68-99dc-6e545ddf4a28 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:12.331238 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfac80c-8d93-4e68-99dc-6e545ddf4a28 None admin] RBAC: Adding query filter params (role.id=b02170460f054c259e825ab6e35f8537, scope.project.id=197e55aa210c42abbf12e585a968f8c5, user.id=a729c9f9f70c41eab4ccf79f7ed5ff84) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:58:12.331775 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfac80c-8d93-4e68-99dc-6e545ddf4a28 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:12.334210 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfac80c-8d93-4e68-99dc-6e545ddf4a28 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:12.336787 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:12.343698 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 86/184] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Sep 6 19:58:12 2022] GET /identity/v3/role_assignments?user.id=a729c9f9f70c41eab4ccf79f7ed5ff84&scope.project.id=197e55aa210c42abbf12e585a968f8c5&role.id=b02170460f054c259e825ab6e35f8537 => generated 608 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:12.344963 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfac80c-8d93-4e68-99dc-6e545ddf4a28 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:12.374357 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baac3a7e-c329-45c4-81cc-c33ad69ac062 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:12.433080 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f313fa-a862-422d-a91b-1955648fbf26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:12.434393 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f313fa-a862-422d-a91b-1955648fbf26 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:12.434733 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f313fa-a862-422d-a91b-1955648fbf26 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:12.435181 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f313fa-a862-422d-a91b-1955648fbf26 None admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:12.435871 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f313fa-a862-422d-a91b-1955648fbf26 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:58:12.436196 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f313fa-a862-422d-a91b-1955648fbf26 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:12.438192 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f313fa-a862-422d-a91b-1955648fbf26 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:12.442772 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:12.458028 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 99/185] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 6 19:58:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:12.458845 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f313fa-a862-422d-a91b-1955648fbf26 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:12.511188 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfac80c-8d93-4e68-99dc-6e545ddf4a28 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:12.569188 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e024d3eb-3cfa-439f-a374-1947c6b7eef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:12.570623 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e024d3eb-3cfa-439f-a374-1947c6b7eef4 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:12.571100 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e024d3eb-3cfa-439f-a374-1947c6b7eef4 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:12.571554 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e024d3eb-3cfa-439f-a374-1947c6b7eef4 None admin] PATH_INFO: `/v3/users/a729c9f9f70c41eab4ccf79f7ed5ff84` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:12.573559 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e024d3eb-3cfa-439f-a374-1947c6b7eef4 None admin] RBAC: Authorizing `identity:get_user(user_id=a729c9f9f70c41eab4ccf79f7ed5ff84)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:12.576601 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e024d3eb-3cfa-439f-a374-1947c6b7eef4 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:12.579291 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 87/186] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 6 19:58:12 2022] GET /identity/v3/users/a729c9f9f70c41eab4ccf79f7ed5ff84 => generated 275 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:12.581254 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e024d3eb-3cfa-439f-a374-1947c6b7eef4 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:12.614505 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f313fa-a862-422d-a91b-1955648fbf26 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:12.676952 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312d0a49-0db7-4e5c-a20b-bc17d8af90b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:12.677658 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312d0a49-0db7-4e5c-a20b-bc17d8af90b6 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:12.677949 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312d0a49-0db7-4e5c-a20b-bc17d8af90b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:12.678262 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312d0a49-0db7-4e5c-a20b-bc17d8af90b6 None admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:12.678948 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312d0a49-0db7-4e5c-a20b-bc17d8af90b6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:58:12.679263 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312d0a49-0db7-4e5c-a20b-bc17d8af90b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:12.681010 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312d0a49-0db7-4e5c-a20b-bc17d8af90b6 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:12.694195 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 100/187] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 6 19:58:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:12.700064 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312d0a49-0db7-4e5c-a20b-bc17d8af90b6 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:12.700279 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:12.760369 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e024d3eb-3cfa-439f-a374-1947c6b7eef4 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:12.787661 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:12.806703 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ef8502-b299-4323-96f0-6eb9f499c6ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:12.807382 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ef8502-b299-4323-96f0-6eb9f499c6ed None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:12.807672 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ef8502-b299-4323-96f0-6eb9f499c6ed None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:12.807949 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ef8502-b299-4323-96f0-6eb9f499c6ed None admin] PATH_INFO: `/v3/domains/default` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:12.809306 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ef8502-b299-4323-96f0-6eb9f499c6ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:12.810928 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ef8502-b299-4323-96f0-6eb9f499c6ed None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:12.812833 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 88/188] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 19:58:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:12.813586 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ef8502-b299-4323-96f0-6eb9f499c6ed None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:12.847273 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312d0a49-0db7-4e5c-a20b-bc17d8af90b6 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:12.893570 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f3a369-1ec6-4b15-ad78-1185822c84b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:12.895303 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f3a369-1ec6-4b15-ad78-1185822c84b0 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:12.895780 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f3a369-1ec6-4b15-ad78-1185822c84b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:12.896232 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f3a369-1ec6-4b15-ad78-1185822c84b0 None admin] PATH_INFO: `/v3/users/20f486167f614910a2e622ee8a5f36fc` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:12.916956 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f3a369-1ec6-4b15-ad78-1185822c84b0 None admin] RBAC: Authorizing `identity:get_user(user_id=20f486167f614910a2e622ee8a5f36fc)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:12.919769 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f3a369-1ec6-4b15-ad78-1185822c84b0 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:12.927012 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 101/189] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 6 19:58:12 2022] GET /identity/v3/users/20f486167f614910a2e622ee8a5f36fc => generated 283 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:12.927012 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f3a369-1ec6-4b15-ad78-1185822c84b0 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:12.950830 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ef8502-b299-4323-96f0-6eb9f499c6ed None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:13.005501 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463b4bd2-7cbf-469c-ab02-5b97c9402105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:13.005782 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463b4bd2-7cbf-469c-ab02-5b97c9402105 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:13.006007 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463b4bd2-7cbf-469c-ab02-5b97c9402105 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:13.006293 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463b4bd2-7cbf-469c-ab02-5b97c9402105 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:13.009441 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:13.057866 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f3a369-1ec6-4b15-ad78-1185822c84b0 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:13.079120 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:13.091774 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463b4bd2-7cbf-469c-ab02-5b97c9402105 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:13.093281 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463b4bd2-7cbf-469c-ab02-5b97c9402105 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:13.103827 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69ff031-6eeb-4e2e-b3d8-5cfcf415b088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:13.104494 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69ff031-6eeb-4e2e-b3d8-5cfcf415b088 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:13.104779 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69ff031-6eeb-4e2e-b3d8-5cfcf415b088 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:13.105057 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69ff031-6eeb-4e2e-b3d8-5cfcf415b088 None admin] PATH_INFO: `/v3/users/20f486167f614910a2e622ee8a5f36fc` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:13.106365 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69ff031-6eeb-4e2e-b3d8-5cfcf415b088 None admin] RBAC: Authorizing `identity:get_user(user_id=20f486167f614910a2e622ee8a5f36fc)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:13.108104 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69ff031-6eeb-4e2e-b3d8-5cfcf415b088 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:13.115718 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 102/190] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 6 19:58:12 2022] GET /identity/v3/users/20f486167f614910a2e622ee8a5f36fc => generated 283 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:13.115718 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69ff031-6eeb-4e2e-b3d8-5cfcf415b088 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:13.222304 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 89/191] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 6 19:58:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 19:58:13.224229 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463b4bd2-7cbf-469c-ab02-5b97c9402105 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:13.240655 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69ff031-6eeb-4e2e-b3d8-5cfcf415b088 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:13.275293 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:13.297353 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f38a8da-1956-4b4f-bfe0-ef13b0698be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:13.298410 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f38a8da-1956-4b4f-bfe0-ef13b0698be2 None admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:13.298890 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f38a8da-1956-4b4f-bfe0-ef13b0698be2 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:13.299358 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f38a8da-1956-4b4f-bfe0-ef13b0698be2 None admin] PATH_INFO: `/v3/users` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:13.300659 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f38a8da-1956-4b4f-bfe0-ef13b0698be2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:13.302924 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f38a8da-1956-4b4f-bfe0-ef13b0698be2 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:13.335448 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 103/192] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Sep 6 19:58:13 2022] POST /identity/v3/users => generated 297 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 19:58:13.343430 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f38a8da-1956-4b4f-bfe0-ef13b0698be2 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:13.412815 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463b4bd2-7cbf-469c-ab02-5b97c9402105 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:13.416767 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:13.468861 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7970fc8-2b9f-4d51-98e7-963a9bab4cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:13.470022 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7970fc8-2b9f-4d51-98e7-963a9bab4cb1 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:13.470599 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7970fc8-2b9f-4d51-98e7-963a9bab4cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:13.471025 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7970fc8-2b9f-4d51-98e7-963a9bab4cb1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:13.520627 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f38a8da-1956-4b4f-bfe0-ef13b0698be2 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:13.602845 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30875a0e-7320-44eb-916a-746be89643ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:13.604086 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30875a0e-7320-44eb-916a-746be89643ad None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:13.605380 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30875a0e-7320-44eb-916a-746be89643ad None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:13.605380 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30875a0e-7320-44eb-916a-746be89643ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:13.609761 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7970fc8-2b9f-4d51-98e7-963a9bab4cb1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:13.613874 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7970fc8-2b9f-4d51-98e7-963a9bab4cb1 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:13.674185 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:13.741750 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30875a0e-7320-44eb-916a-746be89643ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:13.743679 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30875a0e-7320-44eb-916a-746be89643ad None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:13.805492 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 90/193] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 6 19:58:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 582 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 19:58:13.806183 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7970fc8-2b9f-4d51-98e7-963a9bab4cb1 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:13.824915 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:13.970506 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 104/194] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 6 19:58:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 633 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 19:58:13.970506 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30875a0e-7320-44eb-916a-746be89643ad None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:14.050090 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7970fc8-2b9f-4d51-98e7-963a9bab4cb1 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:14.144863 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bbc2bb-58f6-442a-bf8c-7cffc5e3e9bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:14.146024 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bbc2bb-58f6-442a-bf8c-7cffc5e3e9bc None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:14.146521 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bbc2bb-58f6-442a-bf8c-7cffc5e3e9bc None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:14.147008 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bbc2bb-58f6-442a-bf8c-7cffc5e3e9bc None admin] PATH_INFO: `/v3/projects/183ec5fbd9a04da1ab46d9eaccdf955a` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:14.148525 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bbc2bb-58f6-442a-bf8c-7cffc5e3e9bc None admin] RBAC: Authorizing `identity:get_project(project_id=183ec5fbd9a04da1ab46d9eaccdf955a)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:14.151074 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bbc2bb-58f6-442a-bf8c-7cffc5e3e9bc None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:14.153332 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 91/195] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 6 19:58:13 2022] GET /identity/v3/projects/183ec5fbd9a04da1ab46d9eaccdf955a => generated 312 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:14.154685 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bbc2bb-58f6-442a-bf8c-7cffc5e3e9bc None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:14.168375 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:14.189477 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30875a0e-7320-44eb-916a-746be89643ad None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:14.258096 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b0fa7b-63c9-4da8-833b-fc15579af533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:14.259085 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b0fa7b-63c9-4da8-833b-fc15579af533 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:14.259812 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b0fa7b-63c9-4da8-833b-fc15579af533 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:14.260243 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b0fa7b-63c9-4da8-833b-fc15579af533 None admin] PATH_INFO: `/v3/projects/22a00d6dfd004be4aa2f9727f02e69ab` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:14.262486 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b0fa7b-63c9-4da8-833b-fc15579af533 None admin] RBAC: Authorizing `identity:get_project(project_id=22a00d6dfd004be4aa2f9727f02e69ab)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:14.265268 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b0fa7b-63c9-4da8-833b-fc15579af533 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:14.268508 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 105/196] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 6 19:58:13 2022] GET /identity/v3/projects/22a00d6dfd004be4aa2f9727f02e69ab => generated 302 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:14.275022 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb236bd5-5fba-4bf4-9716-ee049477a0ad None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:14.275463 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb236bd5-5fba-4bf4-9716-ee049477a0ad None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:14.275902 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb236bd5-5fba-4bf4-9716-ee049477a0ad None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:14.282572 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 106/197] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 19:58:14 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 19:58:14.284185 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb236bd5-5fba-4bf4-9716-ee049477a0ad None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:14.307713 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:14.313856 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bbc2bb-58f6-442a-bf8c-7cffc5e3e9bc None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:14.354921 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13a2030-98ce-4314-a1e7-9c95cea3b253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:14.356178 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13a2030-98ce-4314-a1e7-9c95cea3b253 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:14.356852 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13a2030-98ce-4314-a1e7-9c95cea3b253 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:14.357341 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13a2030-98ce-4314-a1e7-9c95cea3b253 None admin] PATH_INFO: `/v3/projects/22a00d6dfd004be4aa2f9727f02e69ab` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:14.358938 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13a2030-98ce-4314-a1e7-9c95cea3b253 None admin] RBAC: Authorizing `identity:get_project(project_id=22a00d6dfd004be4aa2f9727f02e69ab)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:14.360655 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13a2030-98ce-4314-a1e7-9c95cea3b253 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:14.363478 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 92/198] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 6 19:58:14 2022] GET /identity/v3/projects/22a00d6dfd004be4aa2f9727f02e69ab => generated 302 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:14.365376 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6287e667-f935-4efd-b44a-899d683cb6cd None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:14.366027 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6287e667-f935-4efd-b44a-899d683cb6cd None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:14.366527 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6287e667-f935-4efd-b44a-899d683cb6cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:14.388975 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-6287e667-f935-4efd-b44a-899d683cb6cd None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 19:58:14.390657 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-6287e667-f935-4efd-b44a-899d683cb6cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 19:58:14.479266 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 93/199] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 19:58:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 19:58:14.480331 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:14.481909 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6287e667-f935-4efd-b44a-899d683cb6cd None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:14.506148 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb236bd5-5fba-4bf4-9716-ee049477a0ad None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:14.574571 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c04c98d-3e6d-4964-b174-0e447938088b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:14.575582 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c04c98d-3e6d-4964-b174-0e447938088b None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:14.576078 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c04c98d-3e6d-4964-b174-0e447938088b None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:14.576576 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c04c98d-3e6d-4964-b174-0e447938088b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:14.578047 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c04c98d-3e6d-4964-b174-0e447938088b None admin] RBAC: Adding query filter params (role.id=4eea39e24b06431aa357053fe7907a3d, scope.project.id=183ec5fbd9a04da1ab46d9eaccdf955a, user.id=a729c9f9f70c41eab4ccf79f7ed5ff84) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:58:14.578621 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c04c98d-3e6d-4964-b174-0e447938088b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:14.581928 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c04c98d-3e6d-4964-b174-0e447938088b None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:14.593815 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 107/200] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Sep 6 19:58:14 2022] GET /identity/v3/role_assignments?user.id=a729c9f9f70c41eab4ccf79f7ed5ff84&scope.project.id=183ec5fbd9a04da1ab46d9eaccdf955a&role.id=4eea39e24b06431aa357053fe7907a3d => generated 608 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:14.595251 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c04c98d-3e6d-4964-b174-0e447938088b None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:14.758627 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:14.813441 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c04c98d-3e6d-4964-b174-0e447938088b None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:14.838846 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6287e667-f935-4efd-b44a-899d683cb6cd None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:14.891104 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fd1268-0d63-4f91-b195-08991855201c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:14.892198 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fd1268-0d63-4f91-b195-08991855201c None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:14.892686 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fd1268-0d63-4f91-b195-08991855201c None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:14.893168 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fd1268-0d63-4f91-b195-08991855201c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:14.894835 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fd1268-0d63-4f91-b195-08991855201c None admin] RBAC: Adding query filter params (role.id=e0ab41d8ae22474688b5361bcc75e0f0, scope.project.id=22a00d6dfd004be4aa2f9727f02e69ab, user.id=20f486167f614910a2e622ee8a5f36fc) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:58:14.895393 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fd1268-0d63-4f91-b195-08991855201c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:14.901352 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fd1268-0d63-4f91-b195-08991855201c None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:14.931431 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 108/201] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Sep 6 19:58:14 2022] GET /identity/v3/role_assignments?user.id=20f486167f614910a2e622ee8a5f36fc&scope.project.id=22a00d6dfd004be4aa2f9727f02e69ab&role.id=e0ab41d8ae22474688b5361bcc75e0f0 => generated 259 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:14.933131 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fd1268-0d63-4f91-b195-08991855201c None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:14.985296 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b0767e-6cd3-4761-b3df-13c95c9f8235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:14.986477 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b0767e-6cd3-4761-b3df-13c95c9f8235 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:14.986969 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b0767e-6cd3-4761-b3df-13c95c9f8235 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:14.987464 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b0767e-6cd3-4761-b3df-13c95c9f8235 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:14.994203 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b0767e-6cd3-4761-b3df-13c95c9f8235 None admin] RBAC: Adding query filter params (role.id=b02170460f054c259e825ab6e35f8537, scope.project.id=22a00d6dfd004be4aa2f9727f02e69ab, user.id=20f486167f614910a2e622ee8a5f36fc) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:58:14.994787 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b0767e-6cd3-4761-b3df-13c95c9f8235 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:15.001546 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b0767e-6cd3-4761-b3df-13c95c9f8235 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:15.028927 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 94/202] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Sep 6 19:58:14 2022] GET /identity/v3/role_assignments?user.id=20f486167f614910a2e622ee8a5f36fc&scope.project.id=22a00d6dfd004be4aa2f9727f02e69ab&role.id=b02170460f054c259e825ab6e35f8537 => generated 259 bytes in 549 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:15.196139 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fd1268-0d63-4f91-b195-08991855201c None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:15.251571 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d86d0a3-3465-449c-99c5-b8e0119cfd49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:15.252535 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d86d0a3-3465-449c-99c5-b8e0119cfd49 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:15.253049 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d86d0a3-3465-449c-99c5-b8e0119cfd49 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:15.253596 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d86d0a3-3465-449c-99c5-b8e0119cfd49 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:15.266844 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:15.274718 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d86d0a3-3465-449c-99c5-b8e0119cfd49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 19:58:15.282138 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-2d86d0a3-3465-449c-99c5-b8e0119cfd49 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 19:58:15.282138 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:58:15.282138 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:58:15.282138 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:58:15.282138 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:58:15.282138 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:58:15.282138 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:58:15.282138 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:58:15.282138 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:58:15.282138 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:58:15.282138 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:58:15.282138 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:58:15.282138 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 19:58:15.282138 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 19:58:15.282138 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 19:58:15.282138 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 19:58:15.283275 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 19:58:15.283275 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 19:58:15.283275 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 19:58:15.283275 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 19:58:15.283275 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:58:15.283275 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:58:15.283275 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:58:15.283275 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:58:15.283275 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:58:15.283275 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:58:15.283275 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:58:15.283275 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 19:58:15.283275 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:58:15.283275 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:58:15.283275 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:58:15.283275 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:58:15.284837 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:58:15.284837 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:58:15.284837 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:58:15.284837 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:58:15.284837 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 19:58:15.284837 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 19:58:15.284837 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 19:58:15.284837 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 19:58:15.284837 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 19:58:15.284837 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 19:58:15.284837 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 19:58:15.284837 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 19:58:15.284837 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 109/203] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 6 19:58:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 352 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:58:15.294175 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b0767e-6cd3-4761-b3df-13c95c9f8235 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:15.420398 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:15.488603 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b0767e-6cd3-4761-b3df-13c95c9f8235 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:15.558749 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bf9fac-347b-44fb-857e-761dfb0a32a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:15.559753 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bf9fac-347b-44fb-857e-761dfb0a32a0 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:15.560201 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bf9fac-347b-44fb-857e-761dfb0a32a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:15.560919 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bf9fac-347b-44fb-857e-761dfb0a32a0 None admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:15.561899 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bf9fac-347b-44fb-857e-761dfb0a32a0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:58:15.562402 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bf9fac-347b-44fb-857e-761dfb0a32a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:15.564895 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bf9fac-347b-44fb-857e-761dfb0a32a0 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:15.584269 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 95/204] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Sep 6 19:58:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:15.591246 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d86d0a3-3465-449c-99c5-b8e0119cfd49 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:15.804165 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d86d0a3-3465-449c-99c5-b8e0119cfd49 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:15.871530 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf0ffb6-e34f-43c5-8974-86d2f48f2d5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:15.872635 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf0ffb6-e34f-43c5-8974-86d2f48f2d5c None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:15.873145 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf0ffb6-e34f-43c5-8974-86d2f48f2d5c None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:15.873662 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf0ffb6-e34f-43c5-8974-86d2f48f2d5c None admin] PATH_INFO: `/v3/users/37e092fe6c824b3cb149b714145f1710` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:15.879151 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf0ffb6-e34f-43c5-8974-86d2f48f2d5c None admin] RBAC: Authorizing `identity:get_user(user_id=37e092fe6c824b3cb149b714145f1710)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:15.881918 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf0ffb6-e34f-43c5-8974-86d2f48f2d5c None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:15.885543 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 110/205] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 6 19:58:15 2022] GET /identity/v3/users/37e092fe6c824b3cb149b714145f1710 => generated 289 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:15.892640 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bf9fac-347b-44fb-857e-761dfb0a32a0 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:16.079775 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bf9fac-347b-44fb-857e-761dfb0a32a0 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:16.129342 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:16.150750 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c1446f-1916-4d73-803c-0a7bba20e837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:16.151695 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c1446f-1916-4d73-803c-0a7bba20e837 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:16.152107 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c1446f-1916-4d73-803c-0a7bba20e837 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:16.152517 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c1446f-1916-4d73-803c-0a7bba20e837 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:16.333283 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c1446f-1916-4d73-803c-0a7bba20e837 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:16.336777 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c1446f-1916-4d73-803c-0a7bba20e837 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:16.612541 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 96/206] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 6 19:58:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 720 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 19:58:16.622894 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf0ffb6-e34f-43c5-8974-86d2f48f2d5c None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:16.747834 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:16.800902 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf0ffb6-e34f-43c5-8974-86d2f48f2d5c None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:16.858517 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059fa2be-76d2-4a98-af28-4e32817a7dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:16.859194 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059fa2be-76d2-4a98-af28-4e32817a7dd5 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:16.859479 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059fa2be-76d2-4a98-af28-4e32817a7dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:16.859760 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059fa2be-76d2-4a98-af28-4e32817a7dd5 None admin] PATH_INFO: `/v3/projects/197e55aa210c42abbf12e585a968f8c5` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:16.861104 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059fa2be-76d2-4a98-af28-4e32817a7dd5 None admin] RBAC: Authorizing `identity:get_project(project_id=197e55aa210c42abbf12e585a968f8c5)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:16.863667 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059fa2be-76d2-4a98-af28-4e32817a7dd5 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:16.865979 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 111/207] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 6 19:58:16 2022] GET /identity/v3/projects/197e55aa210c42abbf12e585a968f8c5 => generated 298 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:16.875875 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c1446f-1916-4d73-803c-0a7bba20e837 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:16.962541 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:17.084521 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c1446f-1916-4d73-803c-0a7bba20e837 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:17.116211 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f436136d-9130-47fb-b02b-cd696dcb4375 None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:17.116436 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f436136d-9130-47fb-b02b-cd696dcb4375 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:17.116737 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f436136d-9130-47fb-b02b-cd696dcb4375 None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:17.119027 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 112/208] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 19:58:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 19:58:17.125729 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71324a91-8250-4e3a-917f-3cec0feb5a95 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:17.126249 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71324a91-8250-4e3a-917f-3cec0feb5a95 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:17.126535 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71324a91-8250-4e3a-917f-3cec0feb5a95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:17.168260 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afd5831-0b40-4fcc-9221-22d4cd24fe20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:17.170740 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afd5831-0b40-4fcc-9221-22d4cd24fe20 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:17.171078 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afd5831-0b40-4fcc-9221-22d4cd24fe20 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:17.171306 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afd5831-0b40-4fcc-9221-22d4cd24fe20 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:17.172095 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afd5831-0b40-4fcc-9221-22d4cd24fe20 None admin] RBAC: Adding query filter params (role.id=034691b44a684a7b8b9bf9f601dd64e9, scope.project.id=197e55aa210c42abbf12e585a968f8c5, user.id=37e092fe6c824b3cb149b714145f1710) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:58:17.173353 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-71324a91-8250-4e3a-917f-3cec0feb5a95 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 19:58:17.174023 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afd5831-0b40-4fcc-9221-22d4cd24fe20 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:17.175770 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afd5831-0b40-4fcc-9221-22d4cd24fe20 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:17.189655 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 97/209] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Sep 6 19:58:16 2022] GET /identity/v3/role_assignments?user.id=37e092fe6c824b3cb149b714145f1710&scope.project.id=197e55aa210c42abbf12e585a968f8c5&role.id=034691b44a684a7b8b9bf9f601dd64e9 => generated 608 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:17.189872 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-71324a91-8250-4e3a-917f-3cec0feb5a95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 19:58:17.295279 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c4f747-73d2-4874-af99-14b06108e6ad None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:17.296038 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c4f747-73d2-4874-af99-14b06108e6ad None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:17.296522 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c4f747-73d2-4874-af99-14b06108e6ad None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:17.298655 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 98/210] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 19:58:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 19:58:17.305600 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0ae38f-e5a5-4a31-9362-9af4cf5db94d None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:17.306277 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0ae38f-e5a5-4a31-9362-9af4cf5db94d None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:17.306753 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0ae38f-e5a5-4a31-9362-9af4cf5db94d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:17.354210 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 113/211] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 19:58:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 19:58:17.367562 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-6b0ae38f-e5a5-4a31-9362-9af4cf5db94d None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 19:58:17.369405 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-6b0ae38f-e5a5-4a31-9362-9af4cf5db94d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 19:58:17.498424 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 99/212] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 19:58:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 19:58:17.726924 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71324a91-8250-4e3a-917f-3cec0feb5a95 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:17.812796 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0ae38f-e5a5-4a31-9362-9af4cf5db94d None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:17.821071 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:17.855207 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:17.889117 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71324a91-8250-4e3a-917f-3cec0feb5a95 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:17.943363 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d466fccb-4610-45e2-aa77-4d0767d85090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:17.944429 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d466fccb-4610-45e2-aa77-4d0767d85090 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:17.944914 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d466fccb-4610-45e2-aa77-4d0767d85090 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:17.945397 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d466fccb-4610-45e2-aa77-4d0767d85090 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:17.947057 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d466fccb-4610-45e2-aa77-4d0767d85090 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:17.949013 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d466fccb-4610-45e2-aa77-4d0767d85090 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:17.951566 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 114/213] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 19:58:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:17.961162 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d466fccb-4610-45e2-aa77-4d0767d85090 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:17.993318 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0ae38f-e5a5-4a31-9362-9af4cf5db94d None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:18.071654 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed37463-9bb3-48c7-8748-893d2321857d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:18.073058 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed37463-9bb3-48c7-8748-893d2321857d None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:18.073598 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed37463-9bb3-48c7-8748-893d2321857d None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:18.074186 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed37463-9bb3-48c7-8748-893d2321857d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:18.098278 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed37463-9bb3-48c7-8748-893d2321857d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 19:58:18.109533 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-bed37463-9bb3-48c7-8748-893d2321857d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 19:58:18.109533 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:58:18.109533 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:58:18.109533 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:58:18.109533 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:58:18.109533 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:58:18.109533 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:58:18.109533 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:58:18.109533 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:58:18.109533 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:58:18.109533 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:58:18.109533 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:58:18.109533 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 19:58:18.109533 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 19:58:18.109533 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 19:58:18.109533 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 19:58:18.111369 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 19:58:18.111369 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 19:58:18.111369 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 19:58:18.111369 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 19:58:18.111369 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:58:18.111369 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:58:18.111369 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:58:18.111369 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:58:18.111369 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:58:18.111369 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:58:18.111369 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:58:18.111369 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 19:58:18.111369 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:58:18.111369 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:58:18.111369 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:58:18.111369 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:58:18.121796 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:58:18.121796 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:58:18.121796 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:58:18.121796 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:58:18.121796 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 19:58:18.121796 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 19:58:18.121796 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 19:58:18.121796 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 19:58:18.121796 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 19:58:18.121796 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 19:58:18.121796 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 19:58:18.121796 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 19:58:18.121796 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 100/214] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 6 19:58:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:58:18.127546 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed37463-9bb3-48c7-8748-893d2321857d None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:18.179590 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d466fccb-4610-45e2-aa77-4d0767d85090 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:18.235096 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:18.243871 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff37babc-8c1b-4665-8c41-329ab47df79a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:18.244736 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff37babc-8c1b-4665-8c41-329ab47df79a None admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:18.245384 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff37babc-8c1b-4665-8c41-329ab47df79a None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:18.245688 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff37babc-8c1b-4665-8c41-329ab47df79a None admin] PATH_INFO: `/v3/users` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:18.246909 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff37babc-8c1b-4665-8c41-329ab47df79a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:18.249553 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff37babc-8c1b-4665-8c41-329ab47df79a None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:18.298215 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 115/215] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Sep 6 19:58:17 2022] POST /identity/v3/users => generated 297 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 19:58:18.306208 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:18.347365 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed37463-9bb3-48c7-8748-893d2321857d None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:18.418288 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe338e28-a204-450c-b4c0-14ade83ba0ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:18.419858 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe338e28-a204-450c-b4c0-14ade83ba0ad None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:18.420698 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe338e28-a204-450c-b4c0-14ade83ba0ad None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:18.421448 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe338e28-a204-450c-b4c0-14ade83ba0ad None admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:18.422898 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe338e28-a204-450c-b4c0-14ade83ba0ad None admin] RBAC: Adding query filter params (name=member) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:58:18.423636 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe338e28-a204-450c-b4c0-14ade83ba0ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:18.426462 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe338e28-a204-450c-b4c0-14ade83ba0ad None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:18.446667 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 101/216] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Sep 6 19:58:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 19:58:18.453583 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff37babc-8c1b-4665-8c41-329ab47df79a None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:18.636236 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff37babc-8c1b-4665-8c41-329ab47df79a None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:18.701202 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366393f9-cff6-4104-b480-cc450dc9edde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:18.702398 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366393f9-cff6-4104-b480-cc450dc9edde None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:18.703012 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366393f9-cff6-4104-b480-cc450dc9edde None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:18.703447 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366393f9-cff6-4104-b480-cc450dc9edde None admin] PATH_INFO: `/v3/users/13a971f1c5f4466f9c62b4d14669c385` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:18.724246 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366393f9-cff6-4104-b480-cc450dc9edde None admin] RBAC: Authorizing `identity:get_user(user_id=13a971f1c5f4466f9c62b4d14669c385)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:18.726092 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366393f9-cff6-4104-b480-cc450dc9edde None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:18.728602 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 116/217] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 6 19:58:18 2022] GET /identity/v3/users/13a971f1c5f4466f9c62b4d14669c385 => generated 297 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:18.735887 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe338e28-a204-450c-b4c0-14ade83ba0ad None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:18.753454 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f70688-17bd-4009-89f4-24402f98356e None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:18.754109 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f70688-17bd-4009-89f4-24402f98356e None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:18.754586 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f70688-17bd-4009-89f4-24402f98356e None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:18.756457 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 117/218] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 19:58:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 19:58:18.765724 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4cbc5b-ba89-4a0c-8400-7b749a473b72 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:18.766240 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4cbc5b-ba89-4a0c-8400-7b749a473b72 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:18.766660 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4cbc5b-ba89-4a0c-8400-7b749a473b72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:18.789099 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-eb4cbc5b-ba89-4a0c-8400-7b749a473b72 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 19:58:18.790670 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-eb4cbc5b-ba89-4a0c-8400-7b749a473b72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 19:58:18.847167 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:18.881361 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 118/219] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 19:58:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 19:58:18.934803 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe338e28-a204-450c-b4c0-14ade83ba0ad None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:18.978298 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4daf4b-7286-4bad-a80b-409aea0f8fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:18.979045 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4daf4b-7286-4bad-a80b-409aea0f8fc5 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:18.979320 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4daf4b-7286-4bad-a80b-409aea0f8fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:18.979603 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4daf4b-7286-4bad-a80b-409aea0f8fc5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:18.996159 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:19.072165 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4daf4b-7286-4bad-a80b-409aea0f8fc5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:19.074071 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4daf4b-7286-4bad-a80b-409aea0f8fc5 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:19.227156 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 102/220] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 6 19:58:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 06 19:58:19.233188 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4cbc5b-ba89-4a0c-8400-7b749a473b72 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:19.299314 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da5831a-2386-47a0-b717-e53ad556d44b None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:19.299655 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da5831a-2386-47a0-b717-e53ad556d44b None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:19.299882 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da5831a-2386-47a0-b717-e53ad556d44b None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:19.301499 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 103/221] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 19:58:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 19:58:19.309310 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925564f7-11fa-4432-85d0-887e084109a9 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:19.309310 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925564f7-11fa-4432-85d0-887e084109a9 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:19.309310 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925564f7-11fa-4432-85d0-887e084109a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:19.330096 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-925564f7-11fa-4432-85d0-887e084109a9 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 19:58:19.332317 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-925564f7-11fa-4432-85d0-887e084109a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 19:58:19.376498 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:19.425348 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 104/222] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 19:58:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 19:58:19.427026 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925564f7-11fa-4432-85d0-887e084109a9 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:19.475014 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4cbc5b-ba89-4a0c-8400-7b749a473b72 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:19.542980 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b74bf3-e21a-4705-8430-e7e1352a80e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:19.544425 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b74bf3-e21a-4705-8430-e7e1352a80e5 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:19.545171 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b74bf3-e21a-4705-8430-e7e1352a80e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:19.545912 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b74bf3-e21a-4705-8430-e7e1352a80e5 None admin] PATH_INFO: `/v3/projects/22a00d6dfd004be4aa2f9727f02e69ab` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:19.548455 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b74bf3-e21a-4705-8430-e7e1352a80e5 None admin] RBAC: Authorizing `identity:get_project(project_id=22a00d6dfd004be4aa2f9727f02e69ab)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:19.551696 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b74bf3-e21a-4705-8430-e7e1352a80e5 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:19.555610 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 119/223] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 6 19:58:19 2022] GET /identity/v3/projects/22a00d6dfd004be4aa2f9727f02e69ab => generated 302 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:19.567364 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b74bf3-e21a-4705-8430-e7e1352a80e5 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:19.677826 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925564f7-11fa-4432-85d0-887e084109a9 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:19.742117 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:19.754074 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42cfcca-cfc0-4d5c-b79b-2088bc7710aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:19.755637 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42cfcca-cfc0-4d5c-b79b-2088bc7710aa None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:19.756758 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42cfcca-cfc0-4d5c-b79b-2088bc7710aa None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:19.757558 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42cfcca-cfc0-4d5c-b79b-2088bc7710aa None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:19.779821 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42cfcca-cfc0-4d5c-b79b-2088bc7710aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 19:58:19.789808 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-e42cfcca-cfc0-4d5c-b79b-2088bc7710aa None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 19:58:19.789808 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:58:19.789808 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:58:19.789808 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:58:19.789808 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:58:19.789808 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:58:19.789808 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:58:19.789808 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:58:19.789808 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:58:19.789808 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:58:19.789808 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:58:19.789808 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:58:19.789808 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 19:58:19.789808 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 19:58:19.789808 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 19:58:19.789808 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 19:58:19.791217 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 19:58:19.791217 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 19:58:19.791217 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 19:58:19.791217 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 19:58:19.791217 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:58:19.791217 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:58:19.791217 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:58:19.791217 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:58:19.791217 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:58:19.791217 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:58:19.791217 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:58:19.791217 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 19:58:19.791217 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:58:19.791217 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:58:19.791217 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:58:19.791217 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:58:19.792490 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:58:19.792490 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:58:19.792490 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:58:19.792490 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:58:19.792490 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 19:58:19.792490 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 19:58:19.792490 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 19:58:19.792490 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 19:58:19.792490 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 19:58:19.792490 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 19:58:19.792490 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 19:58:19.792490 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 19:58:19.796413 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 105/224] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Sep 6 19:58:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 370 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:58:19.798350 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42cfcca-cfc0-4d5c-b79b-2088bc7710aa None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:19.854654 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:19.861187 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b74bf3-e21a-4705-8430-e7e1352a80e5 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:19.927330 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd10689-0d7c-4014-9aa4-6bf90e7f14cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:19.928569 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd10689-0d7c-4014-9aa4-6bf90e7f14cc None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:19.929105 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd10689-0d7c-4014-9aa4-6bf90e7f14cc None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:19.929660 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd10689-0d7c-4014-9aa4-6bf90e7f14cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:19.931623 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd10689-0d7c-4014-9aa4-6bf90e7f14cc None admin] RBAC: Adding query filter params (role.id=4eea39e24b06431aa357053fe7907a3d, scope.project.id=22a00d6dfd004be4aa2f9727f02e69ab, user.id=13a971f1c5f4466f9c62b4d14669c385) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:58:19.931832 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd10689-0d7c-4014-9aa4-6bf90e7f14cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:19.936745 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd10689-0d7c-4014-9aa4-6bf90e7f14cc None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:19.945223 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 120/225] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Sep 6 19:58:19 2022] GET /identity/v3/role_assignments?user.id=13a971f1c5f4466f9c62b4d14669c385&scope.project.id=22a00d6dfd004be4aa2f9727f02e69ab&role.id=4eea39e24b06431aa357053fe7907a3d => generated 259 bytes in 379 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:19.946981 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd10689-0d7c-4014-9aa4-6bf90e7f14cc None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:20.001654 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42cfcca-cfc0-4d5c-b79b-2088bc7710aa None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:20.078233 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93604932-876e-4ceb-8a6c-4a0c1411dcee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:20.079936 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93604932-876e-4ceb-8a6c-4a0c1411dcee None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:20.080839 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93604932-876e-4ceb-8a6c-4a0c1411dcee None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:20.081568 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93604932-876e-4ceb-8a6c-4a0c1411dcee None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:20.123050 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93604932-876e-4ceb-8a6c-4a0c1411dcee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 19:58:20.133051 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-93604932-876e-4ceb-8a6c-4a0c1411dcee None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 19:58:20.133051 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:58:20.133051 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:58:20.133051 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:58:20.133051 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:58:20.133051 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:58:20.133051 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:58:20.133051 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:58:20.133051 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:58:20.133051 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:58:20.133051 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:58:20.133051 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:58:20.133051 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 19:58:20.133051 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 19:58:20.133051 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 19:58:20.133051 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 19:58:20.134709 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 19:58:20.134709 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 19:58:20.134709 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 19:58:20.134709 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 19:58:20.134709 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:58:20.134709 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:58:20.134709 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:58:20.134709 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:58:20.134709 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:58:20.134709 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:58:20.134709 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:58:20.134709 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 19:58:20.134709 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:58:20.134709 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:58:20.134709 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:58:20.134709 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:58:20.136204 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:58:20.136204 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:58:20.136204 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:58:20.136204 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:58:20.136204 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 19:58:20.136204 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 19:58:20.136204 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 19:58:20.136204 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 19:58:20.136204 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 19:58:20.136204 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 19:58:20.136204 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 19:58:20.136204 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 19:58:20.149116 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 106/226] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Sep 6 19:58:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 352 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:58:20.163361 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93604932-876e-4ceb-8a6c-4a0c1411dcee None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:20.207468 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd10689-0d7c-4014-9aa4-6bf90e7f14cc None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:20.227730 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:20.290189 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ecae4a-1f98-42c1-a826-4dca132a048f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:20.291862 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ecae4a-1f98-42c1-a826-4dca132a048f None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:20.292145 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ecae4a-1f98-42c1-a826-4dca132a048f None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:20.292418 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ecae4a-1f98-42c1-a826-4dca132a048f None admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:20.294253 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ecae4a-1f98-42c1-a826-4dca132a048f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:58:20.294620 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ecae4a-1f98-42c1-a826-4dca132a048f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:20.296079 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ecae4a-1f98-42c1-a826-4dca132a048f None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:20.316732 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 121/227] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 6 19:58:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 371 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:20.323075 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ecae4a-1f98-42c1-a826-4dca132a048f None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:20.394252 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:20.409959 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93604932-876e-4ceb-8a6c-4a0c1411dcee None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:20.487280 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8db034-04a7-4380-acf0-bab989ef9400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:20.489201 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8db034-04a7-4380-acf0-bab989ef9400 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:20.490081 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8db034-04a7-4380-acf0-bab989ef9400 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:20.490852 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8db034-04a7-4380-acf0-bab989ef9400 None admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:20.492433 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8db034-04a7-4380-acf0-bab989ef9400 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:58:20.493279 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8db034-04a7-4380-acf0-bab989ef9400 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:20.496575 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8db034-04a7-4380-acf0-bab989ef9400 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:20.514753 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 107/228] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 6 19:58:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:20.522337 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8db034-04a7-4380-acf0-bab989ef9400 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:20.625638 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ecae4a-1f98-42c1-a826-4dca132a048f None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:20.722503 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d726d68-4cfd-46bc-a0a5-c78fbab875ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:20.723102 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d726d68-4cfd-46bc-a0a5-c78fbab875ab None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:20.723466 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d726d68-4cfd-46bc-a0a5-c78fbab875ab None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:20.723749 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d726d68-4cfd-46bc-a0a5-c78fbab875ab None admin] PATH_INFO: `/v3/users/20f486167f614910a2e622ee8a5f36fc` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:20.730572 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d726d68-4cfd-46bc-a0a5-c78fbab875ab None admin] RBAC: Authorizing `identity:get_user(user_id=20f486167f614910a2e622ee8a5f36fc)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:20.732135 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d726d68-4cfd-46bc-a0a5-c78fbab875ab None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:20.734308 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 122/229] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 6 19:58:20 2022] GET /identity/v3/users/20f486167f614910a2e622ee8a5f36fc => generated 283 bytes in 412 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:20.743294 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d726d68-4cfd-46bc-a0a5-c78fbab875ab None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:20.812745 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8db034-04a7-4380-acf0-bab989ef9400 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:20.848829 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:20.890090 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078dc968-6f7f-4302-8503-4cdab548ed59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:20.891210 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078dc968-6f7f-4302-8503-4cdab548ed59 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:20.891686 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078dc968-6f7f-4302-8503-4cdab548ed59 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:20.892354 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078dc968-6f7f-4302-8503-4cdab548ed59 None admin] PATH_INFO: `/v3/users/20f486167f614910a2e622ee8a5f36fc` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:20.894815 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078dc968-6f7f-4302-8503-4cdab548ed59 None admin] RBAC: Authorizing `identity:get_user(user_id=20f486167f614910a2e622ee8a5f36fc)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:20.897815 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078dc968-6f7f-4302-8503-4cdab548ed59 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:20.902847 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 108/230] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 6 19:58:20 2022] GET /identity/v3/users/20f486167f614910a2e622ee8a5f36fc => generated 283 bytes in 381 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:20.911619 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078dc968-6f7f-4302-8503-4cdab548ed59 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:20.961817 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d726d68-4cfd-46bc-a0a5-c78fbab875ab None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:20.982234 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:21.005533 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c62e2f-5991-4066-9553-124f80890bbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:21.006664 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c62e2f-5991-4066-9553-124f80890bbc None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:21.007185 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c62e2f-5991-4066-9553-124f80890bbc None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:21.007693 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c62e2f-5991-4066-9553-124f80890bbc None admin] PATH_INFO: `/v3/projects/22a00d6dfd004be4aa2f9727f02e69ab` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:21.009306 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c62e2f-5991-4066-9553-124f80890bbc None admin] RBAC: Authorizing `identity:get_project(project_id=22a00d6dfd004be4aa2f9727f02e69ab)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:21.011541 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c62e2f-5991-4066-9553-124f80890bbc None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:21.015119 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 123/231] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 6 19:58:20 2022] GET /identity/v3/projects/22a00d6dfd004be4aa2f9727f02e69ab => generated 302 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 19:58:21.023130 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c62e2f-5991-4066-9553-124f80890bbc None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:21.139554 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078dc968-6f7f-4302-8503-4cdab548ed59 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:21.183706 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c62e2f-5991-4066-9553-124f80890bbc None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:21.194375 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:21.227946 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70c5753-0705-4f85-b318-7df19e1ccc1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:21.229246 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70c5753-0705-4f85-b318-7df19e1ccc1d None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:21.229763 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70c5753-0705-4f85-b318-7df19e1ccc1d None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:21.230256 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70c5753-0705-4f85-b318-7df19e1ccc1d None admin] PATH_INFO: `/v3/projects/22a00d6dfd004be4aa2f9727f02e69ab` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:21.233145 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dbe435-f711-4606-acbc-cb2c2e57b62d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:21.233343 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70c5753-0705-4f85-b318-7df19e1ccc1d None admin] RBAC: Authorizing `identity:get_project(project_id=22a00d6dfd004be4aa2f9727f02e69ab)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:21.234339 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dbe435-f711-4606-acbc-cb2c2e57b62d None admin] REQUEST_METHOD: `PUT` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:21.234917 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dbe435-f711-4606-acbc-cb2c2e57b62d None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:21.235573 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dbe435-f711-4606-acbc-cb2c2e57b62d None admin] PATH_INFO: `/v3/projects/22a00d6dfd004be4aa2f9727f02e69ab/users/20f486167f614910a2e622ee8a5f36fc/roles/b02170460f054c259e825ab6e35f8537` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:21.236539 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70c5753-0705-4f85-b318-7df19e1ccc1d None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:21.240071 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dbe435-f711-4606-acbc-cb2c2e57b62d None admin] RBAC: Authorizing `identity:create_grant(project_id=22a00d6dfd004be4aa2f9727f02e69ab, user_id=20f486167f614910a2e622ee8a5f36fc, role_id=b02170460f054c259e825ab6e35f8537)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:21.240466 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 109/232] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 6 19:58:20 2022] GET /identity/v3/projects/22a00d6dfd004be4aa2f9727f02e69ab => generated 302 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:21.242487 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dbe435-f711-4606-acbc-cb2c2e57b62d None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:21.251508 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70c5753-0705-4f85-b318-7df19e1ccc1d None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:21.266814 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 124/233] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Sep 6 19:58:21 2022] PUT /identity/v3/projects/22a00d6dfd004be4aa2f9727f02e69ab/users/20f486167f614910a2e622ee8a5f36fc/roles/b02170460f054c259e825ab6e35f8537 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 19:58:21.460226 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:21.483640 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70c5753-0705-4f85-b318-7df19e1ccc1d None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:21.553510 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa9dcf0-4219-4d95-8b6f-b0051db2791c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:21.554361 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa9dcf0-4219-4d95-8b6f-b0051db2791c None admin] REQUEST_METHOD: `PUT` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:21.555158 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa9dcf0-4219-4d95-8b6f-b0051db2791c None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:21.555449 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa9dcf0-4219-4d95-8b6f-b0051db2791c None admin] PATH_INFO: `/v3/projects/22a00d6dfd004be4aa2f9727f02e69ab/users/20f486167f614910a2e622ee8a5f36fc/roles/e0ab41d8ae22474688b5361bcc75e0f0` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:21.563712 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa9dcf0-4219-4d95-8b6f-b0051db2791c None admin] RBAC: Authorizing `identity:create_grant(project_id=22a00d6dfd004be4aa2f9727f02e69ab, user_id=20f486167f614910a2e622ee8a5f36fc, role_id=e0ab41d8ae22474688b5361bcc75e0f0)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:21.566060 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa9dcf0-4219-4d95-8b6f-b0051db2791c None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:21.591278 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 110/234] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Sep 6 19:58:21 2022] PUT /identity/v3/projects/22a00d6dfd004be4aa2f9727f02e69ab/users/20f486167f614910a2e622ee8a5f36fc/roles/e0ab41d8ae22474688b5361bcc75e0f0 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 19:58:22.312400 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cf34cf-e3fd-40f0-8619-0673c7a05940 None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:22.314652 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cf34cf-e3fd-40f0-8619-0673c7a05940 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:22.315452 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cf34cf-e3fd-40f0-8619-0673c7a05940 None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:22.318456 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 125/235] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 19:58:22 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 19:58:22.325803 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3760618d-9aaa-4add-bcca-89c5e47a736c None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:22.327250 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3760618d-9aaa-4add-bcca-89c5e47a736c None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:22.328026 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3760618d-9aaa-4add-bcca-89c5e47a736c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:22.362978 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-3760618d-9aaa-4add-bcca-89c5e47a736c None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 19:58:22.365744 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-3760618d-9aaa-4add-bcca-89c5e47a736c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 19:58:22.479226 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a56bcc8-5ef4-403e-9042-02ff96b44665 None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:22.480220 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a56bcc8-5ef4-403e-9042-02ff96b44665 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:22.480992 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a56bcc8-5ef4-403e-9042-02ff96b44665 None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:22.484030 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 126/236] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 19:58:22 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 19:58:22.491264 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620113ab-e1ae-4846-9b96-e941f3b2486f None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:22.492211 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620113ab-e1ae-4846-9b96-e941f3b2486f None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:22.492917 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620113ab-e1ae-4846-9b96-e941f3b2486f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:22.539946 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-620113ab-e1ae-4846-9b96-e941f3b2486f None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 19:58:22.542726 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-620113ab-e1ae-4846-9b96-e941f3b2486f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 19:58:22.550539 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 111/237] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 19:58:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 19:58:22.719590 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 127/238] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 19:58:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 19:58:22.979246 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3760618d-9aaa-4add-bcca-89c5e47a736c None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:23.226804 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:23.278889 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3760618d-9aaa-4add-bcca-89c5e47a736c None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:23.372570 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a694da-ec39-432c-8b0d-4dafb04ac634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:23.373745 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a694da-ec39-432c-8b0d-4dafb04ac634 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:23.374181 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a694da-ec39-432c-8b0d-4dafb04ac634 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:23.374666 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a694da-ec39-432c-8b0d-4dafb04ac634 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:23.383169 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a694da-ec39-432c-8b0d-4dafb04ac634 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:23.386239 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a694da-ec39-432c-8b0d-4dafb04ac634 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:23.389773 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 112/239] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 19:58:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 413 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:23.399524 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620113ab-e1ae-4846-9b96-e941f3b2486f None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:23.430323 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a694da-ec39-432c-8b0d-4dafb04ac634 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:23.733273 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a694da-ec39-432c-8b0d-4dafb04ac634 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:23.750177 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:23.769647 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620113ab-e1ae-4846-9b96-e941f3b2486f None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:23.825000 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b052c3-2c80-4f9b-b163-6c5aafe9fa23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:23.826020 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b052c3-2c80-4f9b-b163-6c5aafe9fa23 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:23.826495 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b052c3-2c80-4f9b-b163-6c5aafe9fa23 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:23.826947 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b052c3-2c80-4f9b-b163-6c5aafe9fa23 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:23.861239 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b052c3-2c80-4f9b-b163-6c5aafe9fa23 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 19:58:23.875593 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-33b052c3-2c80-4f9b-b163-6c5aafe9fa23 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 19:58:23.875593 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:58:23.875593 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:58:23.875593 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:58:23.875593 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:58:23.875593 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:58:23.875593 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:58:23.875593 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:58:23.875593 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:58:23.875593 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:58:23.875593 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:58:23.875593 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:58:23.875593 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 19:58:23.875593 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 19:58:23.875593 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 19:58:23.875593 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 19:58:23.877969 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 19:58:23.877969 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 19:58:23.877969 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 19:58:23.877969 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 19:58:23.877969 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:58:23.877969 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:58:23.877969 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:58:23.877969 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:58:23.877969 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:58:23.877969 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:58:23.877969 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:58:23.877969 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 19:58:23.877969 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:58:23.877969 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:58:23.877969 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:58:23.877969 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:58:23.879584 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:58:23.879584 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:58:23.879584 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:58:23.879584 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:58:23.879584 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 19:58:23.879584 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 19:58:23.879584 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 19:58:23.879584 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 19:58:23.879584 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 19:58:23.879584 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 19:58:23.879584 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 19:58:23.879584 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 19:58:23.883830 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 113/240] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 6 19:58:23 2022] GET /identity/v3/roles/reader => generated 84 bytes in 455 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:58:23.889539 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff6b921-e1db-4584-b87a-c6c03ac35f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:23.890363 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff6b921-e1db-4584-b87a-c6c03ac35f34 None admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:23.890872 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff6b921-e1db-4584-b87a-c6c03ac35f34 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:23.891366 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff6b921-e1db-4584-b87a-c6c03ac35f34 None admin] PATH_INFO: `/v3/users` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:23.893091 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff6b921-e1db-4584-b87a-c6c03ac35f34 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:23.894247 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b052c3-2c80-4f9b-b163-6c5aafe9fa23 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:23.896100 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff6b921-e1db-4584-b87a-c6c03ac35f34 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:23.927331 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:23.954890 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 128/241] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Sep 6 19:58:23 2022] POST /identity/v3/users => generated 293 bytes in 557 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 19:58:23.964241 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39b77d6-14d0-4e72-9ccf-46f5f511ed5f None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:23.964749 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39b77d6-14d0-4e72-9ccf-46f5f511ed5f None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:23.965219 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39b77d6-14d0-4e72-9ccf-46f5f511ed5f None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:23.968674 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 129/242] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 19:58:23 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 19:58:23.978643 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099e55a5-955b-4c5a-9e03-274acb8d6a9e None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:23.979288 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099e55a5-955b-4c5a-9e03-274acb8d6a9e None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:23.979774 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099e55a5-955b-4c5a-9e03-274acb8d6a9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:24.034768 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-099e55a5-955b-4c5a-9e03-274acb8d6a9e None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 19:58:24.036927 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-099e55a5-955b-4c5a-9e03-274acb8d6a9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 19:58:24.120451 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b052c3-2c80-4f9b-b163-6c5aafe9fa23 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:24.195870 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b75087-24c7-47e6-810c-f4b304e690ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:24.197597 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b75087-24c7-47e6-810c-f4b304e690ec None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:24.198542 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b75087-24c7-47e6-810c-f4b304e690ec None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:24.199429 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b75087-24c7-47e6-810c-f4b304e690ec None admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:24.201023 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b75087-24c7-47e6-810c-f4b304e690ec None admin] RBAC: Adding query filter params (name=reader) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:58:24.201892 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b75087-24c7-47e6-810c-f4b304e690ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:24.205590 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b75087-24c7-47e6-810c-f4b304e690ec None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:24.224479 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 114/243] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Sep 6 19:58:23 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 19:58:24.236632 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b75087-24c7-47e6-810c-f4b304e690ec None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:24.290738 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 130/244] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 19:58:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 19:58:24.418560 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:24.488670 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b75087-24c7-47e6-810c-f4b304e690ec None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:24.560527 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12224e8-665d-47c6-99a5-55202428e472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:24.561599 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12224e8-665d-47c6-99a5-55202428e472 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:24.562085 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12224e8-665d-47c6-99a5-55202428e472 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:24.562558 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12224e8-665d-47c6-99a5-55202428e472 None admin] PATH_INFO: `/v3/users/ca91686b85ce4371ae69b9caef53fa00` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:24.593262 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12224e8-665d-47c6-99a5-55202428e472 None admin] RBAC: Authorizing `identity:get_user(user_id=ca91686b85ce4371ae69b9caef53fa00)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:24.596765 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12224e8-665d-47c6-99a5-55202428e472 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:24.599903 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 115/245] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 6 19:58:24 2022] GET /identity/v3/users/ca91686b85ce4371ae69b9caef53fa00 => generated 297 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:24.608474 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099e55a5-955b-4c5a-9e03-274acb8d6a9e None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:24.803548 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:24.846663 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099e55a5-955b-4c5a-9e03-274acb8d6a9e None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:24.917218 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12224e8-665d-47c6-99a5-55202428e472 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:24.920448 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd37c27-8da9-47ee-9680-cf28d430e630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:24.921586 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd37c27-8da9-47ee-9680-cf28d430e630 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:24.922122 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd37c27-8da9-47ee-9680-cf28d430e630 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:24.922606 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd37c27-8da9-47ee-9680-cf28d430e630 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:25.074627 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd37c27-8da9-47ee-9680-cf28d430e630 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:25.077814 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd37c27-8da9-47ee-9680-cf28d430e630 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:25.163150 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12224e8-665d-47c6-99a5-55202428e472 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:25.226703 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:25.234773 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43eaa1b-7884-4f3d-a8b9-4bf44ee1d410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:25.235910 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43eaa1b-7884-4f3d-a8b9-4bf44ee1d410 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:25.236381 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43eaa1b-7884-4f3d-a8b9-4bf44ee1d410 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:25.236844 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43eaa1b-7884-4f3d-a8b9-4bf44ee1d410 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:25.249913 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:25.259557 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43eaa1b-7884-4f3d-a8b9-4bf44ee1d410 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 19:58:25.268892 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-a43eaa1b-7884-4f3d-a8b9-4bf44ee1d410 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 19:58:25.268892 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:58:25.268892 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:58:25.268892 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:58:25.268892 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:58:25.268892 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:58:25.268892 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:58:25.268892 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:58:25.268892 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:58:25.268892 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:58:25.268892 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:58:25.268892 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:58:25.268892 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 19:58:25.268892 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 19:58:25.268892 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 19:58:25.268892 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 19:58:25.270959 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 19:58:25.270959 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 19:58:25.270959 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 19:58:25.270959 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 19:58:25.270959 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:58:25.270959 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:58:25.270959 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:58:25.270959 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:58:25.270959 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:58:25.270959 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:58:25.270959 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:58:25.270959 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 19:58:25.270959 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:58:25.270959 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:58:25.270959 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:58:25.270959 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:58:25.272790 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:58:25.272790 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:58:25.272790 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:58:25.272790 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:58:25.272790 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 19:58:25.272790 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 19:58:25.272790 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 19:58:25.272790 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 19:58:25.272790 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 19:58:25.272790 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 19:58:25.272790 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 19:58:25.272790 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 19:58:25.272790 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 116/246] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 6 19:58:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:58:25.278895 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43eaa1b-7884-4f3d-a8b9-4bf44ee1d410 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:25.307761 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 131/247] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 6 19:58:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 700 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 19:58:25.319957 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd37c27-8da9-47ee-9680-cf28d430e630 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:25.636277 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43eaa1b-7884-4f3d-a8b9-4bf44ee1d410 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:25.653549 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd37c27-8da9-47ee-9680-cf28d430e630 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:25.719998 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6bd222-a2f4-423e-8be3-5fff1daf6982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:25.721077 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6bd222-a2f4-423e-8be3-5fff1daf6982 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:25.721573 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6bd222-a2f4-423e-8be3-5fff1daf6982 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:25.722219 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6bd222-a2f4-423e-8be3-5fff1daf6982 None admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:25.723494 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6bd222-a2f4-423e-8be3-5fff1daf6982 None admin] RBAC: Adding query filter params (name=member) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:58:25.723973 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6bd222-a2f4-423e-8be3-5fff1daf6982 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:25.726508 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6bd222-a2f4-423e-8be3-5fff1daf6982 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:25.744623 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 117/248] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Sep 6 19:58:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 466 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:25.747262 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04662399-b204-4836-8f84-f912cec5a12e None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:25.747843 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04662399-b204-4836-8f84-f912cec5a12e None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:25.748311 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04662399-b204-4836-8f84-f912cec5a12e None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:25.751578 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 118/249] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 19:58:25 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 19:58:25.753507 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04662399-b204-4836-8f84-f912cec5a12e None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:25.774951 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11161fe7-add8-4ed1-9dd0-bbe43cb0f3f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:25.776006 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11161fe7-add8-4ed1-9dd0-bbe43cb0f3f8 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:25.776469 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11161fe7-add8-4ed1-9dd0-bbe43cb0f3f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:25.776925 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11161fe7-add8-4ed1-9dd0-bbe43cb0f3f8 None admin] PATH_INFO: `/v3/projects/22a00d6dfd004be4aa2f9727f02e69ab` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:25.783118 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11161fe7-add8-4ed1-9dd0-bbe43cb0f3f8 None admin] RBAC: Authorizing `identity:get_project(project_id=22a00d6dfd004be4aa2f9727f02e69ab)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:25.785886 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11161fe7-add8-4ed1-9dd0-bbe43cb0f3f8 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:25.789153 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 132/250] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 6 19:58:25 2022] GET /identity/v3/projects/22a00d6dfd004be4aa2f9727f02e69ab => generated 302 bytes in 470 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 19:58:25.791219 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f66bea-d970-4618-900d-f0b31da65ca2 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:25.791678 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f66bea-d970-4618-900d-f0b31da65ca2 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:25.792116 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f66bea-d970-4618-900d-f0b31da65ca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:25.857204 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-d7f66bea-d970-4618-900d-f0b31da65ca2 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 19:58:25.859405 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-d7f66bea-d970-4618-900d-f0b31da65ca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 19:58:25.926128 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:25.996797 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04662399-b204-4836-8f84-f912cec5a12e None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:26.006279 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:26.045475 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 133/251] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 19:58:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 19:58:26.047071 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f66bea-d970-4618-900d-f0b31da65ca2 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:26.050215 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51f5c28-ad1b-4ce0-8256-27da606390c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:26.051337 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51f5c28-ad1b-4ce0-8256-27da606390c5 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:26.051821 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51f5c28-ad1b-4ce0-8256-27da606390c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:26.052711 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51f5c28-ad1b-4ce0-8256-27da606390c5 None admin] PATH_INFO: `/v3/users/13a971f1c5f4466f9c62b4d14669c385` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:26.055057 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51f5c28-ad1b-4ce0-8256-27da606390c5 None admin] RBAC: Authorizing `identity:get_user(user_id=13a971f1c5f4466f9c62b4d14669c385)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:26.057563 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51f5c28-ad1b-4ce0-8256-27da606390c5 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:26.060168 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 119/252] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 6 19:58:25 2022] GET /identity/v3/users/13a971f1c5f4466f9c62b4d14669c385 => generated 297 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:26.062931 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f3e3ea-8158-445d-af0b-636ffee35e1c None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:26.064391 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f3e3ea-8158-445d-af0b-636ffee35e1c None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:26.065150 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f3e3ea-8158-445d-af0b-636ffee35e1c None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:26.068317 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 120/253] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 19:58:26 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 19:58:26.075141 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f3e3ea-8158-445d-af0b-636ffee35e1c None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:26.253766 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f66bea-d970-4618-900d-f0b31da65ca2 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:26.315603 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f3e3ea-8158-445d-af0b-636ffee35e1c None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:26.315968 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68db8fa-71fe-45a8-a592-d6b06d9e12de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:26.316972 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68db8fa-71fe-45a8-a592-d6b06d9e12de None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:26.317466 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68db8fa-71fe-45a8-a592-d6b06d9e12de None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:26.317949 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68db8fa-71fe-45a8-a592-d6b06d9e12de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:26.319366 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68db8fa-71fe-45a8-a592-d6b06d9e12de None admin] RBAC: Adding query filter params (role.id=034691b44a684a7b8b9bf9f601dd64e9, scope.project.id=22a00d6dfd004be4aa2f9727f02e69ab, user.id=ca91686b85ce4371ae69b9caef53fa00) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:58:26.319941 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68db8fa-71fe-45a8-a592-d6b06d9e12de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:26.322792 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68db8fa-71fe-45a8-a592-d6b06d9e12de None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:26.334449 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 134/254] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Sep 6 19:58:26 2022] GET /identity/v3/role_assignments?user.id=ca91686b85ce4371ae69b9caef53fa00&scope.project.id=22a00d6dfd004be4aa2f9727f02e69ab&role.id=034691b44a684a7b8b9bf9f601dd64e9 => generated 259 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:26.336568 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6800781d-ad2a-4b29-885d-b4f351ffb9a5 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:26.349932 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6800781d-ad2a-4b29-885d-b4f351ffb9a5 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:26.349932 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6800781d-ad2a-4b29-885d-b4f351ffb9a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:26.358811 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:26.407231 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-6800781d-ad2a-4b29-885d-b4f351ffb9a5 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 19:58:26.409316 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-6800781d-ad2a-4b29-885d-b4f351ffb9a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 19:58:26.415182 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef455c96-eb5f-47fd-8088-5a9de9c35a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:26.415867 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef455c96-eb5f-47fd-8088-5a9de9c35a52 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:26.416161 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef455c96-eb5f-47fd-8088-5a9de9c35a52 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:26.416443 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef455c96-eb5f-47fd-8088-5a9de9c35a52 None admin] PATH_INFO: `/v3/projects/22a00d6dfd004be4aa2f9727f02e69ab` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:26.423240 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef455c96-eb5f-47fd-8088-5a9de9c35a52 None admin] RBAC: Authorizing `identity:get_project(project_id=22a00d6dfd004be4aa2f9727f02e69ab)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:26.424765 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef455c96-eb5f-47fd-8088-5a9de9c35a52 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:26.426625 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 121/255] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 6 19:58:26 2022] GET /identity/v3/projects/22a00d6dfd004be4aa2f9727f02e69ab => generated 302 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:26.432822 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef455c96-eb5f-47fd-8088-5a9de9c35a52 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:26.564524 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:26.617937 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 135/256] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 19:58:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 19:58:26.619553 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6800781d-ad2a-4b29-885d-b4f351ffb9a5 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:26.824400 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef455c96-eb5f-47fd-8088-5a9de9c35a52 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:26.833090 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6800781d-ad2a-4b29-885d-b4f351ffb9a5 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:26.872576 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd3da83-9bd4-435e-b280-9c50b65dace6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:26.875242 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd3da83-9bd4-435e-b280-9c50b65dace6 None admin] REQUEST_METHOD: `PUT` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:26.876626 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd3da83-9bd4-435e-b280-9c50b65dace6 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:26.877862 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd3da83-9bd4-435e-b280-9c50b65dace6 None admin] PATH_INFO: `/v3/projects/22a00d6dfd004be4aa2f9727f02e69ab/users/13a971f1c5f4466f9c62b4d14669c385/roles/4eea39e24b06431aa357053fe7907a3d` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:26.883456 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd3da83-9bd4-435e-b280-9c50b65dace6 None admin] RBAC: Authorizing `identity:create_grant(project_id=22a00d6dfd004be4aa2f9727f02e69ab, user_id=13a971f1c5f4466f9c62b4d14669c385, role_id=4eea39e24b06431aa357053fe7907a3d)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:26.885262 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd3da83-9bd4-435e-b280-9c50b65dace6 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:26.921571 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 122/257] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Sep 6 19:58:26 2022] PUT /identity/v3/projects/22a00d6dfd004be4aa2f9727f02e69ab/users/13a971f1c5f4466f9c62b4d14669c385/roles/4eea39e24b06431aa357053fe7907a3d => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 19:58:26.944693 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0bc906-f49f-49b5-bd9a-f3c7278f39dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:26.946380 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0bc906-f49f-49b5-bd9a-f3c7278f39dd None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:26.947102 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0bc906-f49f-49b5-bd9a-f3c7278f39dd None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:26.947631 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0bc906-f49f-49b5-bd9a-f3c7278f39dd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:26.963904 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0bc906-f49f-49b5-bd9a-f3c7278f39dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 19:58:26.971544 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-2a0bc906-f49f-49b5-bd9a-f3c7278f39dd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 19:58:26.971544 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:58:26.971544 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:58:26.971544 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:58:26.971544 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:58:26.971544 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:58:26.971544 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:58:26.971544 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:58:26.971544 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:58:26.971544 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:58:26.971544 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:58:26.971544 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:58:26.971544 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 19:58:26.971544 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 19:58:26.971544 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 19:58:26.971544 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 19:58:26.972582 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 19:58:26.972582 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 19:58:26.972582 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 19:58:26.972582 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 19:58:26.972582 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:58:26.972582 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:58:26.972582 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:58:26.972582 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:58:26.972582 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:58:26.972582 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:58:26.972582 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:58:26.972582 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 19:58:26.972582 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:58:26.972582 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:58:26.972582 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:58:26.972582 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:58:26.973415 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:58:26.973415 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:58:26.973415 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:58:26.973415 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:58:26.973415 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 19:58:26.973415 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 19:58:26.973415 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 19:58:26.973415 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 19:58:26.973415 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 19:58:26.973415 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 19:58:26.973415 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 19:58:26.973415 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 19:58:26.975719 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 136/258] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Sep 6 19:58:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 357 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:58:26.985087 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd3da83-9bd4-435e-b280-9c50b65dace6 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:27.056009 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:27.186811 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd3da83-9bd4-435e-b280-9c50b65dace6 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:27.259980 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0bc906-f49f-49b5-bd9a-f3c7278f39dd None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:27.286908 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbb6d16-94a7-4f7b-b836-fc28fe500a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:27.288032 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbb6d16-94a7-4f7b-b836-fc28fe500a0e None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:27.288538 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbb6d16-94a7-4f7b-b836-fc28fe500a0e None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:27.289030 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbb6d16-94a7-4f7b-b836-fc28fe500a0e None admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:27.293699 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbb6d16-94a7-4f7b-b836-fc28fe500a0e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:58:27.293699 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbb6d16-94a7-4f7b-b836-fc28fe500a0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:27.293699 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbb6d16-94a7-4f7b-b836-fc28fe500a0e None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:27.311111 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 123/259] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 6 19:58:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 19:58:27.322258 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbb6d16-94a7-4f7b-b836-fc28fe500a0e None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:27.356412 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:27.529450 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0bc906-f49f-49b5-bd9a-f3c7278f39dd None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:27.597267 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66723f8-4e8c-49e4-bcc6-489e139b84ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:27.598341 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66723f8-4e8c-49e4-bcc6-489e139b84ed None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:27.598885 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66723f8-4e8c-49e4-bcc6-489e139b84ed None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:27.599295 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66723f8-4e8c-49e4-bcc6-489e139b84ed None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:27.620213 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbb6d16-94a7-4f7b-b836-fc28fe500a0e None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:27.621050 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66723f8-4e8c-49e4-bcc6-489e139b84ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 19:58:27.633530 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-a66723f8-4e8c-49e4-bcc6-489e139b84ed None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 19:58:27.633530 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:58:27.633530 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:58:27.633530 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:58:27.633530 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:58:27.633530 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:58:27.633530 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:58:27.633530 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:58:27.633530 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:58:27.633530 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:58:27.633530 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:58:27.633530 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:58:27.633530 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 19:58:27.633530 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 19:58:27.633530 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 19:58:27.633530 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 19:58:27.635326 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 19:58:27.635326 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 19:58:27.635326 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 19:58:27.635326 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 19:58:27.635326 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:58:27.635326 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:58:27.635326 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:58:27.635326 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:58:27.635326 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:58:27.635326 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:58:27.635326 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:58:27.635326 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 19:58:27.635326 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:58:27.635326 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:58:27.635326 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:58:27.635326 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:58:27.637292 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:58:27.637292 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:58:27.637292 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:58:27.637292 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:58:27.637292 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 19:58:27.637292 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 19:58:27.637292 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 19:58:27.637292 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 19:58:27.637292 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 19:58:27.637292 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 19:58:27.637292 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 19:58:27.637292 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 19:58:27.637292 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 137/260] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Sep 6 19:58:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 377 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 19:58:27.648574 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:27.649386 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66723f8-4e8c-49e4-bcc6-489e139b84ed None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:27.704307 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11320c27-891c-437d-ad47-7004a91ccdf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:27.705923 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11320c27-891c-437d-ad47-7004a91ccdf2 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:27.706710 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11320c27-891c-437d-ad47-7004a91ccdf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:27.707433 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11320c27-891c-437d-ad47-7004a91ccdf2 None admin] PATH_INFO: `/v3/users/20f486167f614910a2e622ee8a5f36fc` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:27.709840 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11320c27-891c-437d-ad47-7004a91ccdf2 None admin] RBAC: Authorizing `identity:get_user(user_id=20f486167f614910a2e622ee8a5f36fc)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:27.712959 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11320c27-891c-437d-ad47-7004a91ccdf2 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:27.720697 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 124/261] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 6 19:58:27 2022] GET /identity/v3/users/20f486167f614910a2e622ee8a5f36fc => generated 283 bytes in 395 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:27.727180 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11320c27-891c-437d-ad47-7004a91ccdf2 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:27.876070 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:27.892678 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66723f8-4e8c-49e4-bcc6-489e139b84ed None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:28.022605 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec397889-1833-4d2a-9905-25df037cdd36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:28.023229 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec397889-1833-4d2a-9905-25df037cdd36 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:28.023513 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec397889-1833-4d2a-9905-25df037cdd36 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:28.023791 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec397889-1833-4d2a-9905-25df037cdd36 None admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:28.024508 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec397889-1833-4d2a-9905-25df037cdd36 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:58:28.024862 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec397889-1833-4d2a-9905-25df037cdd36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:28.026380 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec397889-1833-4d2a-9905-25df037cdd36 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:28.046099 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11320c27-891c-437d-ad47-7004a91ccdf2 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:28.053639 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 138/262] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 6 19:58:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 402 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 19:58:28.056463 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec397889-1833-4d2a-9905-25df037cdd36 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:28.131903 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abd989c-8572-46e4-8c21-b9167921462d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:28.132733 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abd989c-8572-46e4-8c21-b9167921462d None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:28.133241 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abd989c-8572-46e4-8c21-b9167921462d None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:28.133978 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abd989c-8572-46e4-8c21-b9167921462d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:28.297958 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec397889-1833-4d2a-9905-25df037cdd36 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:28.302674 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:28.352415 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe68d8df-d996-48bb-96c1-0a387a1d988a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:28.353133 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe68d8df-d996-48bb-96c1-0a387a1d988a None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:28.353420 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe68d8df-d996-48bb-96c1-0a387a1d988a None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:28.353764 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe68d8df-d996-48bb-96c1-0a387a1d988a None admin] PATH_INFO: `/v3/users/20f486167f614910a2e622ee8a5f36fc` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:28.356516 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe68d8df-d996-48bb-96c1-0a387a1d988a None admin] RBAC: Authorizing `identity:get_user(user_id=20f486167f614910a2e622ee8a5f36fc)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:28.358067 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe68d8df-d996-48bb-96c1-0a387a1d988a None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:28.359783 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 139/263] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 6 19:58:28 2022] GET /identity/v3/users/20f486167f614910a2e622ee8a5f36fc => generated 283 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 19:58:28.367214 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe68d8df-d996-48bb-96c1-0a387a1d988a None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:28.372224 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abd989c-8572-46e4-8c21-b9167921462d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:28.373890 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abd989c-8572-46e4-8c21-b9167921462d None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:28.426235 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:28.612649 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe68d8df-d996-48bb-96c1-0a387a1d988a None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:28.644888 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 125/264] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 6 19:58:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 919 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 06 19:58:28.646704 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f854c101-c077-44e1-b994-1efffe5d192e None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:28.647250 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f854c101-c077-44e1-b994-1efffe5d192e None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:28.648484 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f854c101-c077-44e1-b994-1efffe5d192e None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:28.650900 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 126/265] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 19:58:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 19:58:28.658834 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f854c101-c077-44e1-b994-1efffe5d192e None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:28.685102 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0b867c-02c3-48a1-bd70-b61209a14872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:28.685719 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0b867c-02c3-48a1-bd70-b61209a14872 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:28.686064 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0b867c-02c3-48a1-bd70-b61209a14872 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:28.686350 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0b867c-02c3-48a1-bd70-b61209a14872 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:28.848398 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0b867c-02c3-48a1-bd70-b61209a14872 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:28.850200 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0b867c-02c3-48a1-bd70-b61209a14872 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:28.858448 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:28.939064 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:28.948263 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f854c101-c077-44e1-b994-1efffe5d192e None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:29.026831 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e1faea-7917-4d2a-8903-1ca4e0ac16a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:29.027935 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e1faea-7917-4d2a-8903-1ca4e0ac16a4 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:29.028436 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e1faea-7917-4d2a-8903-1ca4e0ac16a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:29.028935 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e1faea-7917-4d2a-8903-1ca4e0ac16a4 None admin] PATH_INFO: `/v3/projects/22a00d6dfd004be4aa2f9727f02e69ab` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:29.032167 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e1faea-7917-4d2a-8903-1ca4e0ac16a4 None admin] RBAC: Authorizing `identity:get_project(project_id=22a00d6dfd004be4aa2f9727f02e69ab)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:29.036406 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e1faea-7917-4d2a-8903-1ca4e0ac16a4 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:29.040561 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 127/266] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 6 19:58:28 2022] GET /identity/v3/projects/22a00d6dfd004be4aa2f9727f02e69ab => generated 302 bytes in 382 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:29.043044 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b63bab-b455-4802-90b1-b734b03c1dce None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:29.043835 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b63bab-b455-4802-90b1-b734b03c1dce None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:29.044546 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b63bab-b455-4802-90b1-b734b03c1dce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:29.103233 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-42b63bab-b455-4802-90b1-b734b03c1dce None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 19:58:29.106067 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-42b63bab-b455-4802-90b1-b734b03c1dce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 19:58:29.117530 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 140/267] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 6 19:58:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 751 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 19:58:29.123123 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0b867c-02c3-48a1-bd70-b61209a14872 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:29.243353 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 128/268] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 19:58:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 19:58:29.244657 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b63bab-b455-4802-90b1-b734b03c1dce None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:29.265028 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0b867c-02c3-48a1-bd70-b61209a14872 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:29.317234 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:29.345926 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:29.352753 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84590d66-26eb-462c-a9df-3a055a3479f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:29.353944 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84590d66-26eb-462c-a9df-3a055a3479f4 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:29.356307 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84590d66-26eb-462c-a9df-3a055a3479f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:29.357037 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84590d66-26eb-462c-a9df-3a055a3479f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:29.358538 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84590d66-26eb-462c-a9df-3a055a3479f4 None admin] RBAC: Adding query filter params (role.id=b02170460f054c259e825ab6e35f8537, scope.project.id=22a00d6dfd004be4aa2f9727f02e69ab, user.id=20f486167f614910a2e622ee8a5f36fc) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:58:29.360824 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84590d66-26eb-462c-a9df-3a055a3479f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:29.364375 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84590d66-26eb-462c-a9df-3a055a3479f4 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:29.381869 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 141/269] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Sep 6 19:58:29 2022] GET /identity/v3/role_assignments?user.id=20f486167f614910a2e622ee8a5f36fc&scope.project.id=22a00d6dfd004be4aa2f9727f02e69ab&role.id=b02170460f054c259e825ab6e35f8537 => generated 608 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:29.389205 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652b8506-318e-42fb-8cb8-4d4bf11cca22 None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:29.390102 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652b8506-318e-42fb-8cb8-4d4bf11cca22 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:29.390850 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652b8506-318e-42fb-8cb8-4d4bf11cca22 None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:29.396395 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 142/270] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 19:58:29 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 19:58:29.417086 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b63bab-b455-4802-90b1-b734b03c1dce None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:29.425134 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ae1a5f-3c63-4329-9f05-3d278a2e4774 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:29.425859 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ae1a5f-3c63-4329-9f05-3d278a2e4774 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:29.426376 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ae1a5f-3c63-4329-9f05-3d278a2e4774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:29.475613 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-f6ae1a5f-3c63-4329-9f05-3d278a2e4774 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 19:58:29.477950 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-f6ae1a5f-3c63-4329-9f05-3d278a2e4774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 19:58:29.483281 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306e3907-2423-4d15-890c-c7a6f985b25f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:29.486207 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306e3907-2423-4d15-890c-c7a6f985b25f None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:29.486827 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306e3907-2423-4d15-890c-c7a6f985b25f None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:29.487396 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306e3907-2423-4d15-890c-c7a6f985b25f None admin] PATH_INFO: `/v3/projects/22a00d6dfd004be4aa2f9727f02e69ab` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:29.489119 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306e3907-2423-4d15-890c-c7a6f985b25f None admin] RBAC: Authorizing `identity:get_project(project_id=22a00d6dfd004be4aa2f9727f02e69ab)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:29.491500 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306e3907-2423-4d15-890c-c7a6f985b25f None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:29.501947 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 129/271] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 6 19:58:29 2022] GET /identity/v3/projects/22a00d6dfd004be4aa2f9727f02e69ab => generated 302 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:29.508677 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306e3907-2423-4d15-890c-c7a6f985b25f None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:29.603597 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 143/272] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 19:58:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 19:58:29.713536 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306e3907-2423-4d15-890c-c7a6f985b25f None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:29.788629 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7541d56-2eec-4de8-898e-652aeff90830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:29.789822 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7541d56-2eec-4de8-898e-652aeff90830 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:29.790323 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7541d56-2eec-4de8-898e-652aeff90830 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:29.790818 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7541d56-2eec-4de8-898e-652aeff90830 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:29.792678 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7541d56-2eec-4de8-898e-652aeff90830 None admin] RBAC: Adding query filter params (role.id=e0ab41d8ae22474688b5361bcc75e0f0, scope.project.id=22a00d6dfd004be4aa2f9727f02e69ab, user.id=20f486167f614910a2e622ee8a5f36fc) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:58:29.793251 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7541d56-2eec-4de8-898e-652aeff90830 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:29.796149 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7541d56-2eec-4de8-898e-652aeff90830 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:29.802190 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ae1a5f-3c63-4329-9f05-3d278a2e4774 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:29.815399 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 130/273] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Sep 6 19:58:29 2022] GET /identity/v3/role_assignments?user.id=20f486167f614910a2e622ee8a5f36fc&scope.project.id=22a00d6dfd004be4aa2f9727f02e69ab&role.id=e0ab41d8ae22474688b5361bcc75e0f0 => generated 608 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 19:58:29.980525 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ae1a5f-3c63-4329-9f05-3d278a2e4774 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:30.004655 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:30.038123 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7e074b-c5fb-4d50-8c7e-3edae7be518e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:30.039450 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7e074b-c5fb-4d50-8c7e-3edae7be518e None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:30.039953 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7e074b-c5fb-4d50-8c7e-3edae7be518e None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:30.040435 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7e074b-c5fb-4d50-8c7e-3edae7be518e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:30.045851 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7e074b-c5fb-4d50-8c7e-3edae7be518e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:30.048030 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7e074b-c5fb-4d50-8c7e-3edae7be518e None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:30.050616 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 144/274] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 19:58:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:30.058558 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7541d56-2eec-4de8-898e-652aeff90830 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:30.082064 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:30.214458 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7e074b-c5fb-4d50-8c7e-3edae7be518e None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:30.318356 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7541d56-2eec-4de8-898e-652aeff90830 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:30.393534 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33743f8-c1bd-4977-b8ab-2c65b72f2e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:30.394824 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33743f8-c1bd-4977-b8ab-2c65b72f2e72 None admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:30.395395 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33743f8-c1bd-4977-b8ab-2c65b72f2e72 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:30.395883 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33743f8-c1bd-4977-b8ab-2c65b72f2e72 None admin] PATH_INFO: `/v3/users` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:30.397126 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33743f8-c1bd-4977-b8ab-2c65b72f2e72 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:30.400263 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33743f8-c1bd-4977-b8ab-2c65b72f2e72 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:30.417946 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7e074b-c5fb-4d50-8c7e-3edae7be518e None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:30.445811 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 131/275] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Sep 6 19:58:30 2022] POST /identity/v3/users => generated 293 bytes in 385 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 19:58:30.461420 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb493a0-9135-45a7-9518-ae82aec1741a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:30.462285 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb493a0-9135-45a7-9518-ae82aec1741a None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:30.462570 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb493a0-9135-45a7-9518-ae82aec1741a None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:30.462848 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb493a0-9135-45a7-9518-ae82aec1741a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:30.477378 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb493a0-9135-45a7-9518-ae82aec1741a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 19:58:30.482988 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-9fb493a0-9135-45a7-9518-ae82aec1741a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 19:58:30.482988 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:58:30.482988 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:58:30.482988 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:58:30.482988 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:58:30.482988 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:58:30.482988 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:58:30.482988 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:58:30.482988 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:58:30.482988 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:58:30.482988 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:58:30.482988 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:58:30.482988 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 19:58:30.482988 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 19:58:30.482988 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 19:58:30.482988 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 19:58:30.483990 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 19:58:30.483990 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 19:58:30.483990 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 19:58:30.483990 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 19:58:30.483990 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:58:30.483990 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:58:30.483990 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:58:30.483990 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:58:30.483990 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:58:30.483990 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:58:30.483990 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:58:30.483990 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 19:58:30.483990 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:58:30.483990 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:58:30.483990 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:58:30.483990 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:58:30.485052 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:58:30.485052 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:58:30.485052 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:58:30.485052 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:58:30.485052 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 19:58:30.485052 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 19:58:30.485052 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 19:58:30.485052 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 19:58:30.485052 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 19:58:30.485052 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 19:58:30.485052 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 19:58:30.485052 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 19:58:30.485052 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 145/276] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 6 19:58:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:58:30.490681 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33743f8-c1bd-4977-b8ab-2c65b72f2e72 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:30.551319 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c993a7-28ea-466c-9119-12aa9c4f9485 None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:30.551644 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c993a7-28ea-466c-9119-12aa9c4f9485 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:30.551876 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c993a7-28ea-466c-9119-12aa9c4f9485 None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:30.554043 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 146/277] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 19:58:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 19:58:30.569018 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dce9a27-69c2-4829-8052-6ece72b5ed5a None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:30.570780 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dce9a27-69c2-4829-8052-6ece72b5ed5a None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:30.571932 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dce9a27-69c2-4829-8052-6ece72b5ed5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:30.592555 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:30.626109 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-7dce9a27-69c2-4829-8052-6ece72b5ed5a None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 19:58:30.630209 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-7dce9a27-69c2-4829-8052-6ece72b5ed5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 19:58:30.684947 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33743f8-c1bd-4977-b8ab-2c65b72f2e72 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:30.723863 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:30.742646 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9727ce-fdcf-4963-9959-ebe4cb5a6dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:30.743623 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9727ce-fdcf-4963-9959-ebe4cb5a6dfe None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:30.743915 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9727ce-fdcf-4963-9959-ebe4cb5a6dfe None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:30.744194 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9727ce-fdcf-4963-9959-ebe4cb5a6dfe None admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:30.744905 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9727ce-fdcf-4963-9959-ebe4cb5a6dfe None admin] RBAC: Adding query filter params (name=member) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:58:30.745220 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9727ce-fdcf-4963-9959-ebe4cb5a6dfe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:30.749516 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9727ce-fdcf-4963-9959-ebe4cb5a6dfe None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:30.773677 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 132/278] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Sep 6 19:58:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:30.777070 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9727ce-fdcf-4963-9959-ebe4cb5a6dfe None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:30.787249 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 147/279] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 19:58:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 19:58:30.897215 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801d18bd-aee8-4c03-87b8-b2827b461865 None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:30.897559 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801d18bd-aee8-4c03-87b8-b2827b461865 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:30.898086 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801d18bd-aee8-4c03-87b8-b2827b461865 None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:30.899466 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 148/280] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 19:58:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 19:58:30.905727 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93187d24-d6ac-4a2e-8cbe-b009fa1e57f3 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:30.906354 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93187d24-d6ac-4a2e-8cbe-b009fa1e57f3 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:30.907103 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93187d24-d6ac-4a2e-8cbe-b009fa1e57f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:30.934506 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-93187d24-d6ac-4a2e-8cbe-b009fa1e57f3 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 19:58:30.936177 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-93187d24-d6ac-4a2e-8cbe-b009fa1e57f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 19:58:30.962699 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9727ce-fdcf-4963-9959-ebe4cb5a6dfe None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:30.997016 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:31.021176 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e14e4ce-9e15-4047-b619-59812ca16814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:31.025037 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e14e4ce-9e15-4047-b619-59812ca16814 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:31.025534 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e14e4ce-9e15-4047-b619-59812ca16814 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:31.026051 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e14e4ce-9e15-4047-b619-59812ca16814 None admin] PATH_INFO: `/v3/users/c85315ab3f5e418dbcfc1f4071e3a53b` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:31.046451 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e14e4ce-9e15-4047-b619-59812ca16814 None admin] RBAC: Authorizing `identity:get_user(user_id=c85315ab3f5e418dbcfc1f4071e3a53b)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:31.048589 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e14e4ce-9e15-4047-b619-59812ca16814 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:31.050775 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 133/281] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 6 19:58:30 2022] GET /identity/v3/users/c85315ab3f5e418dbcfc1f4071e3a53b => generated 293 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 19:58:31.057324 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e14e4ce-9e15-4047-b619-59812ca16814 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:31.071872 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 149/282] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 19:58:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 19:58:31.262165 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e14e4ce-9e15-4047-b619-59812ca16814 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:31.319847 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a0b50a-398d-48a5-b747-a6354d803cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:31.320822 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a0b50a-398d-48a5-b747-a6354d803cf1 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:31.320822 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a0b50a-398d-48a5-b747-a6354d803cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:31.321057 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a0b50a-398d-48a5-b747-a6354d803cf1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:31.321921 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a0b50a-398d-48a5-b747-a6354d803cf1 None admin] RBAC: Adding query filter params (role.id=4eea39e24b06431aa357053fe7907a3d, user.id=c85315ab3f5e418dbcfc1f4071e3a53b, scope.system=all) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:58:31.322237 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a0b50a-398d-48a5-b747-a6354d803cf1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:31.323846 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a0b50a-398d-48a5-b747-a6354d803cf1 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:31.339223 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 134/283] 10.222.0.20 () {56 vars in 1308 bytes} [Tue Sep 6 19:58:31 2022] GET /identity/v3/role_assignments?user.id=c85315ab3f5e418dbcfc1f4071e3a53b&scope.system=all&role.id=4eea39e24b06431aa357053fe7907a3d => generated 226 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:31.383802 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93187d24-d6ac-4a2e-8cbe-b009fa1e57f3 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:31.620208 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93187d24-d6ac-4a2e-8cbe-b009fa1e57f3 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:31.636320 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:31.691319 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a0b50a-398d-48a5-b747-a6354d803cf1 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:31.715163 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6709a9-4c10-4b3a-bd2e-262438ec2df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:31.716614 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6709a9-4c10-4b3a-bd2e-262438ec2df4 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:31.717297 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6709a9-4c10-4b3a-bd2e-262438ec2df4 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:31.717977 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6709a9-4c10-4b3a-bd2e-262438ec2df4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:31.740097 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6709a9-4c10-4b3a-bd2e-262438ec2df4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 19:58:31.750255 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-7b6709a9-4c10-4b3a-bd2e-262438ec2df4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 19:58:31.750255 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:58:31.750255 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:58:31.750255 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:58:31.750255 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:58:31.750255 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:58:31.750255 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:58:31.750255 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:58:31.750255 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:58:31.750255 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:58:31.750255 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:58:31.750255 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:58:31.750255 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 19:58:31.750255 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 19:58:31.750255 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 19:58:31.750255 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 19:58:31.752168 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 19:58:31.752168 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 19:58:31.752168 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 19:58:31.752168 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 19:58:31.752168 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:58:31.752168 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:58:31.752168 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:58:31.752168 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:58:31.752168 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:58:31.752168 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:58:31.752168 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:58:31.752168 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 19:58:31.752168 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:58:31.752168 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:58:31.752168 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:58:31.752168 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:58:31.753982 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:58:31.753982 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:58:31.753982 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:58:31.753982 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:58:31.753982 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 19:58:31.753982 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 19:58:31.753982 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 19:58:31.753982 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 19:58:31.753982 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 19:58:31.753982 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 19:58:31.753982 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 19:58:31.753982 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 19:58:31.757865 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 150/284] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 6 19:58:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 376 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:58:31.769551 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6709a9-4c10-4b3a-bd2e-262438ec2df4 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:31.842310 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:31.943178 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a0b50a-398d-48a5-b747-a6354d803cf1 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:32.020113 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20827737-32e5-4026-9593-0e1415be001b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:32.021719 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20827737-32e5-4026-9593-0e1415be001b None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:32.022454 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20827737-32e5-4026-9593-0e1415be001b None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:32.023184 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20827737-32e5-4026-9593-0e1415be001b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:32.047805 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20827737-32e5-4026-9593-0e1415be001b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 19:58:32.058418 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-20827737-32e5-4026-9593-0e1415be001b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 19:58:32.058418 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:58:32.058418 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:58:32.058418 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:58:32.058418 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:58:32.058418 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:58:32.058418 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:58:32.058418 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:58:32.058418 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:58:32.058418 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:58:32.058418 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:58:32.058418 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:58:32.058418 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 19:58:32.058418 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 19:58:32.058418 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 19:58:32.058418 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 19:58:32.060149 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 19:58:32.060149 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 19:58:32.060149 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 19:58:32.060149 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 19:58:32.060149 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:58:32.060149 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:58:32.060149 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:58:32.060149 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:58:32.060149 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:58:32.060149 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:58:32.060149 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:58:32.060149 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 19:58:32.060149 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:58:32.060149 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:58:32.060149 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:58:32.060149 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:58:32.061771 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:58:32.061771 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:58:32.061771 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:58:32.061771 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:58:32.061771 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 19:58:32.061771 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 19:58:32.061771 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 19:58:32.061771 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 19:58:32.061771 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 19:58:32.061771 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 19:58:32.061771 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 19:58:32.061771 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 19:58:32.066663 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 135/285] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 6 19:58:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 376 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:58:32.082351 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20827737-32e5-4026-9593-0e1415be001b None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:32.178516 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6709a9-4c10-4b3a-bd2e-262438ec2df4 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:32.285788 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20827737-32e5-4026-9593-0e1415be001b None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:32.302146 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:32.308082 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e54fbdd-b52b-4815-9ff5-850f407b353d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:32.309130 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e54fbdd-b52b-4815-9ff5-850f407b353d None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:32.309636 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e54fbdd-b52b-4815-9ff5-850f407b353d None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:32.310126 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e54fbdd-b52b-4815-9ff5-850f407b353d None admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:32.311397 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e54fbdd-b52b-4815-9ff5-850f407b353d None admin] RBAC: Adding query filter params (name=reader) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:58:32.311935 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e54fbdd-b52b-4815-9ff5-850f407b353d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:32.323297 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e54fbdd-b52b-4815-9ff5-850f407b353d None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:32.347395 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6891c10d-4f4f-4e1a-bd00-61bb2894b0e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:32.348911 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6891c10d-4f4f-4e1a-bd00-61bb2894b0e7 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:32.349565 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6891c10d-4f4f-4e1a-bd00-61bb2894b0e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:32.350372 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6891c10d-4f4f-4e1a-bd00-61bb2894b0e7 None admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:32.352127 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6891c10d-4f4f-4e1a-bd00-61bb2894b0e7 None admin] RBAC: Adding query filter params (name=member) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:58:32.352964 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6891c10d-4f4f-4e1a-bd00-61bb2894b0e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:32.354921 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6891c10d-4f4f-4e1a-bd00-61bb2894b0e7 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:32.358194 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 151/286] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Sep 6 19:58:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 589 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 19:58:32.368301 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 136/287] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Sep 6 19:58:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:32.377841 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e54fbdd-b52b-4815-9ff5-850f407b353d None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:32.381208 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6891c10d-4f4f-4e1a-bd00-61bb2894b0e7 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:32.505168 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:32.532145 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6891c10d-4f4f-4e1a-bd00-61bb2894b0e7 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:32.622423 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f67679e-9d56-419e-8586-b3d229d25532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:32.623092 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f67679e-9d56-419e-8586-b3d229d25532 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:32.623373 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f67679e-9d56-419e-8586-b3d229d25532 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:32.623652 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f67679e-9d56-419e-8586-b3d229d25532 None admin] PATH_INFO: `/v3/users/13a971f1c5f4466f9c62b4d14669c385` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:32.629279 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f67679e-9d56-419e-8586-b3d229d25532 None admin] RBAC: Authorizing `identity:get_user(user_id=13a971f1c5f4466f9c62b4d14669c385)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:32.632116 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f67679e-9d56-419e-8586-b3d229d25532 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:32.635513 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 137/288] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 6 19:58:32 2022] GET /identity/v3/users/13a971f1c5f4466f9c62b4d14669c385 => generated 297 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 19:58:32.657660 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f67679e-9d56-419e-8586-b3d229d25532 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:32.675939 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e54fbdd-b52b-4815-9ff5-850f407b353d None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:32.785411 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410cc13b-1442-4536-be2b-ad4d7f322e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:32.787162 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410cc13b-1442-4536-be2b-ad4d7f322e52 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:32.788021 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410cc13b-1442-4536-be2b-ad4d7f322e52 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:32.788823 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410cc13b-1442-4536-be2b-ad4d7f322e52 None admin] PATH_INFO: `/v3/users/ca91686b85ce4371ae69b9caef53fa00` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:32.800998 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410cc13b-1442-4536-be2b-ad4d7f322e52 None admin] RBAC: Authorizing `identity:get_user(user_id=ca91686b85ce4371ae69b9caef53fa00)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:32.816676 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410cc13b-1442-4536-be2b-ad4d7f322e52 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:32.820863 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:32.823739 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 152/289] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 6 19:58:32 2022] GET /identity/v3/users/ca91686b85ce4371ae69b9caef53fa00 => generated 297 bytes in 455 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:32.838308 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410cc13b-1442-4536-be2b-ad4d7f322e52 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:33.011845 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f67679e-9d56-419e-8586-b3d229d25532 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:33.076506 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32339b4b-169e-4717-9286-1e6b7ed19ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:33.077499 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32339b4b-169e-4717-9286-1e6b7ed19ded None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:33.078025 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32339b4b-169e-4717-9286-1e6b7ed19ded None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:33.079054 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32339b4b-169e-4717-9286-1e6b7ed19ded None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:33.114046 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:33.124140 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410cc13b-1442-4536-be2b-ad4d7f322e52 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:33.191023 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b5b048-2a9c-485e-a810-05b246487977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:33.192274 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b5b048-2a9c-485e-a810-05b246487977 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:33.192891 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b5b048-2a9c-485e-a810-05b246487977 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:33.193513 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b5b048-2a9c-485e-a810-05b246487977 None admin] PATH_INFO: `/v3/projects/22a00d6dfd004be4aa2f9727f02e69ab` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:33.195302 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b5b048-2a9c-485e-a810-05b246487977 None admin] RBAC: Authorizing `identity:get_project(project_id=22a00d6dfd004be4aa2f9727f02e69ab)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:33.197449 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b5b048-2a9c-485e-a810-05b246487977 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:33.200104 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 153/290] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 6 19:58:32 2022] GET /identity/v3/projects/22a00d6dfd004be4aa2f9727f02e69ab => generated 302 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:33.206635 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b5b048-2a9c-485e-a810-05b246487977 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:33.271489 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32339b4b-169e-4717-9286-1e6b7ed19ded None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:33.274453 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32339b4b-169e-4717-9286-1e6b7ed19ded None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:33.374008 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:33.400772 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b5b048-2a9c-485e-a810-05b246487977 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:33.483200 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76592bc9-afab-4f4f-a411-d883e55f6732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:33.483901 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76592bc9-afab-4f4f-a411-d883e55f6732 None admin] REQUEST_METHOD: `PUT` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:33.484182 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76592bc9-afab-4f4f-a411-d883e55f6732 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:33.484561 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76592bc9-afab-4f4f-a411-d883e55f6732 None admin] PATH_INFO: `/v3/projects/22a00d6dfd004be4aa2f9727f02e69ab/users/ca91686b85ce4371ae69b9caef53fa00/roles/034691b44a684a7b8b9bf9f601dd64e9` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:33.488547 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76592bc9-afab-4f4f-a411-d883e55f6732 None admin] RBAC: Authorizing `identity:create_grant(project_id=22a00d6dfd004be4aa2f9727f02e69ab, user_id=ca91686b85ce4371ae69b9caef53fa00, role_id=034691b44a684a7b8b9bf9f601dd64e9)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:33.490589 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76592bc9-afab-4f4f-a411-d883e55f6732 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:33.524857 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 154/291] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Sep 6 19:58:33 2022] PUT /identity/v3/projects/22a00d6dfd004be4aa2f9727f02e69ab/users/ca91686b85ce4371ae69b9caef53fa00/roles/034691b44a684a7b8b9bf9f601dd64e9 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 19:58:33.551160 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 138/292] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 6 19:58:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 897 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 19:58:33.568725 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76592bc9-afab-4f4f-a411-d883e55f6732 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:33.638447 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:33.779331 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76592bc9-afab-4f4f-a411-d883e55f6732 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:33.834589 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddcf67c-2c2e-44e9-abfb-570e56f47098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:33.835402 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddcf67c-2c2e-44e9-abfb-570e56f47098 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:33.835687 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddcf67c-2c2e-44e9-abfb-570e56f47098 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:33.836013 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddcf67c-2c2e-44e9-abfb-570e56f47098 None admin] PATH_INFO: `/v3/projects/22a00d6dfd004be4aa2f9727f02e69ab` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:33.837350 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddcf67c-2c2e-44e9-abfb-570e56f47098 None admin] RBAC: Authorizing `identity:get_project(project_id=22a00d6dfd004be4aa2f9727f02e69ab)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:33.838951 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddcf67c-2c2e-44e9-abfb-570e56f47098 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:33.841136 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 155/293] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 6 19:58:33 2022] GET /identity/v3/projects/22a00d6dfd004be4aa2f9727f02e69ab => generated 302 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:33.850666 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32339b4b-169e-4717-9286-1e6b7ed19ded None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:33.883813 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5cc7a7-435f-4d79-a9fd-532bc1a9289f None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:33.884274 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5cc7a7-435f-4d79-a9fd-532bc1a9289f None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:33.884750 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5cc7a7-435f-4d79-a9fd-532bc1a9289f None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:33.887495 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 156/294] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 19:58:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 19:58:33.897711 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bc85cc-1aca-4673-8ea0-03760bdabc7b None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:33.898085 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bc85cc-1aca-4673-8ea0-03760bdabc7b None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:33.898537 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bc85cc-1aca-4673-8ea0-03760bdabc7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:33.963627 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-65bc85cc-1aca-4673-8ea0-03760bdabc7b None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 19:58:33.965895 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-65bc85cc-1aca-4673-8ea0-03760bdabc7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 19:58:34.080789 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32339b4b-169e-4717-9286-1e6b7ed19ded None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:34.150465 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbb5410-9a93-4aee-94bc-be64dea0967d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:34.151389 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbb5410-9a93-4aee-94bc-be64dea0967d None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:34.151885 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbb5410-9a93-4aee-94bc-be64dea0967d None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:34.152375 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbb5410-9a93-4aee-94bc-be64dea0967d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:34.153759 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbb5410-9a93-4aee-94bc-be64dea0967d None admin] RBAC: Adding query filter params (role.id=4eea39e24b06431aa357053fe7907a3d, scope.project.id=22a00d6dfd004be4aa2f9727f02e69ab, user.id=13a971f1c5f4466f9c62b4d14669c385) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:58:34.154298 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbb5410-9a93-4aee-94bc-be64dea0967d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:34.157222 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbb5410-9a93-4aee-94bc-be64dea0967d None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:34.168998 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 139/295] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Sep 6 19:58:33 2022] GET /identity/v3/role_assignments?user.id=13a971f1c5f4466f9c62b4d14669c385&scope.project.id=22a00d6dfd004be4aa2f9727f02e69ab&role.id=4eea39e24b06431aa357053fe7907a3d => generated 608 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:34.169709 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:34.178369 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 157/296] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 19:58:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 19:58:34.568875 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6bf951-e62d-47f3-8334-f74be7fd4aa2 None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:34.569390 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6bf951-e62d-47f3-8334-f74be7fd4aa2 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:34.569801 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6bf951-e62d-47f3-8334-f74be7fd4aa2 None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:34.572441 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 140/297] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 19:58:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 19:58:34.584405 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7939dcc0-cef0-4e44-90d8-56699f120a39 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:34.586131 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7939dcc0-cef0-4e44-90d8-56699f120a39 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:34.586850 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7939dcc0-cef0-4e44-90d8-56699f120a39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:34.619905 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-7939dcc0-cef0-4e44-90d8-56699f120a39 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 19:58:34.622353 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-7939dcc0-cef0-4e44-90d8-56699f120a39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 19:58:34.654366 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:34.760605 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 158/298] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 19:58:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 19:58:34.777058 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6bf951-e62d-47f3-8334-f74be7fd4aa2 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:35.025071 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6bf951-e62d-47f3-8334-f74be7fd4aa2 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:35.079661 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaed7805-c01d-4d55-9769-2508f84e82ec None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:35.079942 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaed7805-c01d-4d55-9769-2508f84e82ec None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:35.080224 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaed7805-c01d-4d55-9769-2508f84e82ec None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:35.082185 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 159/299] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 19:58:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 19:58:35.088583 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa24c5f-bfaa-49cf-840e-dea881865aec None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:35.089275 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa24c5f-bfaa-49cf-840e-dea881865aec None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:35.089799 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa24c5f-bfaa-49cf-840e-dea881865aec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:35.101557 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07e84f8-3173-4d90-a779-9df93d8f9698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:35.102707 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07e84f8-3173-4d90-a779-9df93d8f9698 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:35.103192 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07e84f8-3173-4d90-a779-9df93d8f9698 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:35.103686 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07e84f8-3173-4d90-a779-9df93d8f9698 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:35.106395 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07e84f8-3173-4d90-a779-9df93d8f9698 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:35.109428 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07e84f8-3173-4d90-a779-9df93d8f9698 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:35.112904 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-eaa24c5f-bfaa-49cf-840e-dea881865aec None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 19:58:35.113085 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 141/300] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 19:58:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 19:58:35.115162 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-eaa24c5f-bfaa-49cf-840e-dea881865aec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 19:58:35.121718 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07e84f8-3173-4d90-a779-9df93d8f9698 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:35.237551 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:35.299579 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 160/301] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 19:58:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 19:58:35.303138 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07e84f8-3173-4d90-a779-9df93d8f9698 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:35.336215 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa24c5f-bfaa-49cf-840e-dea881865aec None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:35.383924 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b11516-50c7-4096-9fee-1831f33e875f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:35.385046 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b11516-50c7-4096-9fee-1831f33e875f None admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:35.385559 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b11516-50c7-4096-9fee-1831f33e875f None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:35.386360 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b11516-50c7-4096-9fee-1831f33e875f None admin] PATH_INFO: `/v3/groups` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:35.387604 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b11516-50c7-4096-9fee-1831f33e875f None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:35.391176 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b11516-50c7-4096-9fee-1831f33e875f None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:35.430958 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 142/302] 10.222.0.20 () {60 vars in 1134 bytes} [Tue Sep 6 19:58:35 2022] POST /identity/v3/groups => generated 229 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 19:58:35.535703 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:35.569769 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa24c5f-bfaa-49cf-840e-dea881865aec None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:35.626486 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc9b1f3-4db8-4dbb-93bb-45ef24125679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:35.627776 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc9b1f3-4db8-4dbb-93bb-45ef24125679 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:35.627776 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc9b1f3-4db8-4dbb-93bb-45ef24125679 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:35.628191 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc9b1f3-4db8-4dbb-93bb-45ef24125679 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:35.648982 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc9b1f3-4db8-4dbb-93bb-45ef24125679 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 19:58:35.655284 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-bdc9b1f3-4db8-4dbb-93bb-45ef24125679 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 19:58:35.655284 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:58:35.655284 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:58:35.655284 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:58:35.655284 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:58:35.655284 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:58:35.655284 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:58:35.655284 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:58:35.655284 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:58:35.655284 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:58:35.655284 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:58:35.655284 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:58:35.655284 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 19:58:35.655284 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 19:58:35.655284 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 19:58:35.655284 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 19:58:35.657363 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 19:58:35.657363 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 19:58:35.657363 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 19:58:35.657363 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 19:58:35.657363 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:58:35.657363 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:58:35.657363 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:58:35.657363 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:58:35.657363 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:58:35.657363 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:58:35.657363 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:58:35.657363 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 19:58:35.657363 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:58:35.657363 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:58:35.657363 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:58:35.657363 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:58:35.659002 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:58:35.659002 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:58:35.659002 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:58:35.659002 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:58:35.659002 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 19:58:35.659002 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 19:58:35.659002 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 19:58:35.659002 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 19:58:35.659002 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 19:58:35.659002 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 19:58:35.659002 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 19:58:35.659002 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 19:58:35.659002 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 161/303] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 6 19:58:35 2022] GET /identity/v3/roles/reader => generated 84 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 19:58:35.671482 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b11516-50c7-4096-9fee-1831f33e875f None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:35.750938 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc9b1f3-4db8-4dbb-93bb-45ef24125679 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:35.895234 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b11516-50c7-4096-9fee-1831f33e875f None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:35.926883 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc9b1f3-4db8-4dbb-93bb-45ef24125679 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:35.952820 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:35.967983 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d4d0d4-209f-43af-b5c9-d51bb21a3b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:35.969763 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d4d0d4-209f-43af-b5c9-d51bb21a3b2f None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:35.970279 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d4d0d4-209f-43af-b5c9-d51bb21a3b2f None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:35.970888 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d4d0d4-209f-43af-b5c9-d51bb21a3b2f None admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:35.972819 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d4d0d4-209f-43af-b5c9-d51bb21a3b2f None admin] RBAC: Adding query filter params (name=reader) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:58:35.973381 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d4d0d4-209f-43af-b5c9-d51bb21a3b2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:35.975462 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d4d0d4-209f-43af-b5c9-d51bb21a3b2f None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:35.994798 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 143/304] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Sep 6 19:58:35 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:35.996559 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d4d0d4-209f-43af-b5c9-d51bb21a3b2f None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:36.007725 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d71131-b742-4335-b91d-79024d6308e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:36.008911 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d71131-b742-4335-b91d-79024d6308e3 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:36.010383 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d71131-b742-4335-b91d-79024d6308e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:36.011176 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d71131-b742-4335-b91d-79024d6308e3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:36.027980 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d71131-b742-4335-b91d-79024d6308e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 19:58:36.034390 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-c2d71131-b742-4335-b91d-79024d6308e3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 19:58:36.034390 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:58:36.034390 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:58:36.034390 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:58:36.034390 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:58:36.034390 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:58:36.034390 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:58:36.034390 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:58:36.034390 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:58:36.034390 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:58:36.034390 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:58:36.034390 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:58:36.034390 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 19:58:36.034390 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 19:58:36.034390 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 19:58:36.034390 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 19:58:36.035377 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 19:58:36.035377 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 19:58:36.035377 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 19:58:36.035377 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 19:58:36.035377 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:58:36.035377 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:58:36.035377 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:58:36.035377 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:58:36.035377 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:58:36.035377 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:58:36.035377 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:58:36.035377 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 19:58:36.035377 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:58:36.035377 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:58:36.035377 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:58:36.035377 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:58:36.036331 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:58:36.036331 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:58:36.036331 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:58:36.036331 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:58:36.036331 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 19:58:36.036331 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 19:58:36.036331 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 19:58:36.036331 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 19:58:36.036331 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 19:58:36.036331 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 19:58:36.036331 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 19:58:36.036331 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 19:58:36.039315 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 162/305] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 6 19:58:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 19:58:36.046099 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d71131-b742-4335-b91d-79024d6308e3 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:36.051302 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:36.147444 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d4d0d4-209f-43af-b5c9-d51bb21a3b2f None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:36.174726 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d71131-b742-4335-b91d-79024d6308e3 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:36.191878 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335d0c4e-a194-4e5c-a3c7-8e1d41e11f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:36.192541 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335d0c4e-a194-4e5c-a3c7-8e1d41e11f2a None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:36.192898 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335d0c4e-a194-4e5c-a3c7-8e1d41e11f2a None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:36.193199 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335d0c4e-a194-4e5c-a3c7-8e1d41e11f2a None admin] PATH_INFO: `/v3/users/8e81688c48174cefb1141af6f553ed82` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:36.215091 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335d0c4e-a194-4e5c-a3c7-8e1d41e11f2a None admin] RBAC: Authorizing `identity:get_user(user_id=8e81688c48174cefb1141af6f553ed82)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:36.217998 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335d0c4e-a194-4e5c-a3c7-8e1d41e11f2a None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:36.218438 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47be3d89-220b-41c4-972c-047312d36d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:36.219670 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47be3d89-220b-41c4-972c-047312d36d31 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:36.220294 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47be3d89-220b-41c4-972c-047312d36d31 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:36.220774 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47be3d89-220b-41c4-972c-047312d36d31 None admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:36.226497 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 144/306] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 6 19:58:35 2022] GET /identity/v3/users/8e81688c48174cefb1141af6f553ed82 => generated 293 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 19:58:36.226686 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47be3d89-220b-41c4-972c-047312d36d31 None admin] RBAC: Adding query filter params (name=member) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:58:36.228515 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335d0c4e-a194-4e5c-a3c7-8e1d41e11f2a None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:36.229518 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47be3d89-220b-41c4-972c-047312d36d31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:36.233301 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47be3d89-220b-41c4-972c-047312d36d31 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:36.269967 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 163/307] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Sep 6 19:58:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:36.273143 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47be3d89-220b-41c4-972c-047312d36d31 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:36.305712 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:36.412035 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335d0c4e-a194-4e5c-a3c7-8e1d41e11f2a None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:36.417913 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:36.485675 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa99e1fc-5199-4348-89ae-9f7c02895b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:36.487791 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa99e1fc-5199-4348-89ae-9f7c02895b44 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:36.488862 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa99e1fc-5199-4348-89ae-9f7c02895b44 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:36.489550 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa99e1fc-5199-4348-89ae-9f7c02895b44 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:36.491408 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa99e1fc-5199-4348-89ae-9f7c02895b44 None admin] RBAC: Adding query filter params (role.id=034691b44a684a7b8b9bf9f601dd64e9, user.id=8e81688c48174cefb1141af6f553ed82, scope.system=all) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:58:36.492763 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa99e1fc-5199-4348-89ae-9f7c02895b44 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:36.496280 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa99e1fc-5199-4348-89ae-9f7c02895b44 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:36.504879 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47be3d89-220b-41c4-972c-047312d36d31 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:36.509319 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 145/308] 10.222.0.20 () {56 vars in 1308 bytes} [Tue Sep 6 19:58:36 2022] GET /identity/v3/role_assignments?user.id=8e81688c48174cefb1141af6f553ed82&scope.system=all&role.id=034691b44a684a7b8b9bf9f601dd64e9 => generated 226 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:36.584783 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9550f80-8ccf-4079-a4af-4b0181933729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:36.586049 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9550f80-8ccf-4079-a4af-4b0181933729 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:36.586551 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9550f80-8ccf-4079-a4af-4b0181933729 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:36.587030 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9550f80-8ccf-4079-a4af-4b0181933729 None admin] PATH_INFO: `/v3/users/c85315ab3f5e418dbcfc1f4071e3a53b` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:36.589376 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9550f80-8ccf-4079-a4af-4b0181933729 None admin] RBAC: Authorizing `identity:get_user(user_id=c85315ab3f5e418dbcfc1f4071e3a53b)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:36.592374 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9550f80-8ccf-4079-a4af-4b0181933729 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:36.601872 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 164/309] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 6 19:58:36 2022] GET /identity/v3/users/c85315ab3f5e418dbcfc1f4071e3a53b => generated 293 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 19:58:36.607032 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa99e1fc-5199-4348-89ae-9f7c02895b44 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:36.798786 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa99e1fc-5199-4348-89ae-9f7c02895b44 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:36.816895 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:36.833856 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b635513-f3ff-43da-a79f-0b71172258cc None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:36.834509 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b635513-f3ff-43da-a79f-0b71172258cc None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:36.834642 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b635513-f3ff-43da-a79f-0b71172258cc None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:36.837239 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 165/310] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 19:58:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 19:58:36.847283 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0afcb3-8f25-416e-a21d-cf112955a33a None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:36.847769 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0afcb3-8f25-416e-a21d-cf112955a33a None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:36.848271 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0afcb3-8f25-416e-a21d-cf112955a33a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:36.850135 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4570aa-8a7f-4667-8025-d794d1bf3232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:36.851091 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4570aa-8a7f-4667-8025-d794d1bf3232 None admin] REQUEST_METHOD: `PUT` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:36.851603 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4570aa-8a7f-4667-8025-d794d1bf3232 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:36.852513 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4570aa-8a7f-4667-8025-d794d1bf3232 None admin] PATH_INFO: `/v3/system/users/c85315ab3f5e418dbcfc1f4071e3a53b/roles/4eea39e24b06431aa357053fe7907a3d` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:36.855074 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4570aa-8a7f-4667-8025-d794d1bf3232 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=c85315ab3f5e418dbcfc1f4071e3a53b, role_id=4eea39e24b06431aa357053fe7907a3d)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:36.856997 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4570aa-8a7f-4667-8025-d794d1bf3232 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:36.871361 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 146/311] 10.222.0.20 () {58 vars in 1334 bytes} [Tue Sep 6 19:58:36 2022] PUT /identity/v3/system/users/c85315ab3f5e418dbcfc1f4071e3a53b/roles/4eea39e24b06431aa357053fe7907a3d => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 19:58:36.889498 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-be0afcb3-8f25-416e-a21d-cf112955a33a None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 19:58:36.893889 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-be0afcb3-8f25-416e-a21d-cf112955a33a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 19:58:36.997513 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 166/312] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 19:58:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 19:58:37.293122 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4570aa-8a7f-4667-8025-d794d1bf3232 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:37.426926 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4570aa-8a7f-4667-8025-d794d1bf3232 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:37.470219 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01c6285-7a65-4c9a-bdff-bad00bee612d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:37.470892 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01c6285-7a65-4c9a-bdff-bad00bee612d None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:37.471329 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01c6285-7a65-4c9a-bdff-bad00bee612d None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:37.471811 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01c6285-7a65-4c9a-bdff-bad00bee612d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:37.486637 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01c6285-7a65-4c9a-bdff-bad00bee612d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 19:58:37.492323 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-a01c6285-7a65-4c9a-bdff-bad00bee612d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 19:58:37.492323 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:58:37.492323 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:58:37.492323 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:58:37.492323 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:58:37.492323 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:58:37.492323 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:58:37.492323 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:58:37.492323 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:58:37.492323 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:58:37.492323 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:58:37.492323 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:58:37.492323 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 19:58:37.492323 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 19:58:37.492323 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 19:58:37.492323 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 19:58:37.493729 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 19:58:37.493729 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 19:58:37.493729 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 19:58:37.493729 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 19:58:37.493729 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:58:37.493729 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:58:37.493729 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:58:37.493729 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:58:37.493729 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:58:37.493729 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:58:37.493729 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:58:37.493729 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 19:58:37.493729 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:58:37.493729 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:58:37.493729 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:58:37.493729 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:58:37.495145 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:58:37.495145 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:58:37.495145 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:58:37.495145 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:58:37.495145 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 19:58:37.495145 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 19:58:37.495145 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 19:58:37.495145 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 19:58:37.495145 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 19:58:37.495145 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 19:58:37.495145 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 19:58:37.495145 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 19:58:37.495145 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 147/313] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 6 19:58:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 19:58:37.503853 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0afcb3-8f25-416e-a21d-cf112955a33a None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:37.630961 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:37.690791 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0afcb3-8f25-416e-a21d-cf112955a33a None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:37.735760 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93a9826-2d21-4804-bcc9-92cb1a923f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:37.736418 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93a9826-2d21-4804-bcc9-92cb1a923f13 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:37.736702 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93a9826-2d21-4804-bcc9-92cb1a923f13 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:37.736976 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93a9826-2d21-4804-bcc9-92cb1a923f13 None admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:37.737733 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93a9826-2d21-4804-bcc9-92cb1a923f13 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:58:37.738045 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93a9826-2d21-4804-bcc9-92cb1a923f13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:37.739513 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93a9826-2d21-4804-bcc9-92cb1a923f13 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:37.757863 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 167/314] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Sep 6 19:58:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 19:58:37.759444 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01c6285-7a65-4c9a-bdff-bad00bee612d None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:37.808198 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:37.907845 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01c6285-7a65-4c9a-bdff-bad00bee612d None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:37.954901 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363f76fc-9ffd-495c-afc6-526e458b28b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:37.955581 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363f76fc-9ffd-495c-afc6-526e458b28b9 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:37.955865 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363f76fc-9ffd-495c-afc6-526e458b28b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:37.956145 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363f76fc-9ffd-495c-afc6-526e458b28b9 None admin] PATH_INFO: `/v3/users/ca91686b85ce4371ae69b9caef53fa00` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:37.957580 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363f76fc-9ffd-495c-afc6-526e458b28b9 None admin] RBAC: Authorizing `identity:get_user(user_id=ca91686b85ce4371ae69b9caef53fa00)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:37.959226 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363f76fc-9ffd-495c-afc6-526e458b28b9 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:37.965799 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 148/315] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 6 19:58:37 2022] GET /identity/v3/users/ca91686b85ce4371ae69b9caef53fa00 => generated 297 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:37.968175 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93a9826-2d21-4804-bcc9-92cb1a923f13 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:38.104051 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93a9826-2d21-4804-bcc9-92cb1a923f13 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:38.137911 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:38.153406 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f0b560-904d-48d5-a837-719869542fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:38.154239 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f0b560-904d-48d5-a837-719869542fb9 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:38.154573 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f0b560-904d-48d5-a837-719869542fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:38.154850 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f0b560-904d-48d5-a837-719869542fb9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:38.283157 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f0b560-904d-48d5-a837-719869542fb9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:38.285771 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f0b560-904d-48d5-a837-719869542fb9 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:38.485247 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 168/316] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 6 19:58:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 517 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 19:58:38.492339 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363f76fc-9ffd-495c-afc6-526e458b28b9 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:38.619264 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363f76fc-9ffd-495c-afc6-526e458b28b9 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:38.622139 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:38.664504 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db08189d-55eb-4dbb-969b-0b3855461a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:38.665160 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db08189d-55eb-4dbb-969b-0b3855461a77 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:38.665442 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db08189d-55eb-4dbb-969b-0b3855461a77 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:38.665765 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db08189d-55eb-4dbb-969b-0b3855461a77 None admin] PATH_INFO: `/v3/projects/22a00d6dfd004be4aa2f9727f02e69ab` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:38.667052 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db08189d-55eb-4dbb-969b-0b3855461a77 None admin] RBAC: Authorizing `identity:get_project(project_id=22a00d6dfd004be4aa2f9727f02e69ab)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:38.668721 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db08189d-55eb-4dbb-969b-0b3855461a77 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:38.675993 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 149/317] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 6 19:58:38 2022] GET /identity/v3/projects/22a00d6dfd004be4aa2f9727f02e69ab => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 19:58:38.679018 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f0b560-904d-48d5-a837-719869542fb9 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:38.723190 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:38.854653 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f0b560-904d-48d5-a837-719869542fb9 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:38.904280 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6e73dc-489a-4cef-a156-9ccfd17b33e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:38.904916 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6e73dc-489a-4cef-a156-9ccfd17b33e9 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:38.905181 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6e73dc-489a-4cef-a156-9ccfd17b33e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:38.905458 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6e73dc-489a-4cef-a156-9ccfd17b33e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:38.906446 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6e73dc-489a-4cef-a156-9ccfd17b33e9 None admin] RBAC: Adding query filter params (role.id=034691b44a684a7b8b9bf9f601dd64e9, scope.project.id=22a00d6dfd004be4aa2f9727f02e69ab, user.id=ca91686b85ce4371ae69b9caef53fa00) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:58:38.906878 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6e73dc-489a-4cef-a156-9ccfd17b33e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:38.909036 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6e73dc-489a-4cef-a156-9ccfd17b33e9 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:38.921060 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 169/318] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Sep 6 19:58:38 2022] GET /identity/v3/role_assignments?user.id=ca91686b85ce4371ae69b9caef53fa00&scope.project.id=22a00d6dfd004be4aa2f9727f02e69ab&role.id=034691b44a684a7b8b9bf9f601dd64e9 => generated 608 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:38.933513 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de0f60c-8bb1-42ef-93ca-b75f6e2a6239 None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:38.934500 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de0f60c-8bb1-42ef-93ca-b75f6e2a6239 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:38.934500 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de0f60c-8bb1-42ef-93ca-b75f6e2a6239 None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:38.936170 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 150/319] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 19:58:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 19:58:38.943454 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faf882d-ea3c-436a-b5b7-77583292e547 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:38.943866 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faf882d-ea3c-436a-b5b7-77583292e547 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:38.944098 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faf882d-ea3c-436a-b5b7-77583292e547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:38.966718 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-1faf882d-ea3c-436a-b5b7-77583292e547 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 19:58:38.968148 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-1faf882d-ea3c-436a-b5b7-77583292e547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 19:58:39.063406 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 170/320] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 19:58:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 19:58:39.354180 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de0f60c-8bb1-42ef-93ca-b75f6e2a6239 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:39.496104 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de0f60c-8bb1-42ef-93ca-b75f6e2a6239 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:39.541784 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a11ea30-e523-4dac-b67a-5b0d47268264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:39.542446 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a11ea30-e523-4dac-b67a-5b0d47268264 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:39.542723 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a11ea30-e523-4dac-b67a-5b0d47268264 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:39.542997 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a11ea30-e523-4dac-b67a-5b0d47268264 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:39.544219 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a11ea30-e523-4dac-b67a-5b0d47268264 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:39.546050 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a11ea30-e523-4dac-b67a-5b0d47268264 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:39.548771 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 151/321] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 19:58:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 19:58:39.555090 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faf882d-ea3c-436a-b5b7-77583292e547 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:39.584777 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:39.753383 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faf882d-ea3c-436a-b5b7-77583292e547 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:39.839974 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03645719-8b5f-49ee-a856-d2e4f94a34cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:39.841141 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03645719-8b5f-49ee-a856-d2e4f94a34cc None admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:39.841670 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03645719-8b5f-49ee-a856-d2e4f94a34cc None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:39.842148 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03645719-8b5f-49ee-a856-d2e4f94a34cc None admin] PATH_INFO: `/v3/groups` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:39.842732 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a61ca78-0e3c-45fd-a31c-f42ac168a661 None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:39.843079 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a61ca78-0e3c-45fd-a31c-f42ac168a661 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:39.843342 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03645719-8b5f-49ee-a856-d2e4f94a34cc None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:39.843536 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a61ca78-0e3c-45fd-a31c-f42ac168a661 None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:39.844840 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 152/322] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 19:58:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 19:58:39.846336 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03645719-8b5f-49ee-a856-d2e4f94a34cc None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:39.850921 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b9e384-6414-444c-8807-e17034f4e389 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:39.851201 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b9e384-6414-444c-8807-e17034f4e389 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:39.851712 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b9e384-6414-444c-8807-e17034f4e389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:39.872493 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-e3b9e384-6414-444c-8807-e17034f4e389 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 19:58:39.873913 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-e3b9e384-6414-444c-8807-e17034f4e389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 19:58:39.885653 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 171/323] 10.222.0.20 () {60 vars in 1134 bytes} [Tue Sep 6 19:58:39 2022] POST /identity/v3/groups => generated 226 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 19:58:39.936516 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:40.003228 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 153/324] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 19:58:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 19:58:40.479673 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa954b9-11c6-4785-a788-7d30da428a73 None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:40.481236 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa954b9-11c6-4785-a788-7d30da428a73 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:40.482040 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa954b9-11c6-4785-a788-7d30da428a73 None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:40.485590 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 172/325] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 19:58:40 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 19:58:40.499079 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056a8256-1825-4ff1-bf84-36f577f322b1 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:40.500076 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056a8256-1825-4ff1-bf84-36f577f322b1 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:40.500867 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056a8256-1825-4ff1-bf84-36f577f322b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:40.516668 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa954b9-11c6-4785-a788-7d30da428a73 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:40.535543 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-056a8256-1825-4ff1-bf84-36f577f322b1 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 19:58:40.537406 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-056a8256-1825-4ff1-bf84-36f577f322b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 19:58:40.715197 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:40.729668 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 154/326] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 19:58:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 19:58:40.766665 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa954b9-11c6-4785-a788-7d30da428a73 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:40.841169 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834a88eb-e651-4366-a9e3-b2c26ee38c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:40.842769 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834a88eb-e651-4366-a9e3-b2c26ee38c0d None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:40.844220 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834a88eb-e651-4366-a9e3-b2c26ee38c0d None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:40.845070 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834a88eb-e651-4366-a9e3-b2c26ee38c0d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:40.867692 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834a88eb-e651-4366-a9e3-b2c26ee38c0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 19:58:40.879786 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-834a88eb-e651-4366-a9e3-b2c26ee38c0d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 19:58:40.879786 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:58:40.879786 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:58:40.879786 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:58:40.879786 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:58:40.879786 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:58:40.879786 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:58:40.879786 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:58:40.879786 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:58:40.879786 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:58:40.879786 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:58:40.879786 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:58:40.879786 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 19:58:40.879786 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 19:58:40.879786 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 19:58:40.879786 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 19:58:40.881637 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 19:58:40.881637 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 19:58:40.881637 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 19:58:40.881637 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 19:58:40.881637 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:58:40.881637 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:58:40.881637 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:58:40.881637 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:58:40.881637 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:58:40.881637 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:58:40.881637 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:58:40.881637 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 19:58:40.881637 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:58:40.881637 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:58:40.881637 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:58:40.881637 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:58:40.883247 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:58:40.883247 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:58:40.883247 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:58:40.883247 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:58:40.883247 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 19:58:40.883247 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 19:58:40.883247 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 19:58:40.883247 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 19:58:40.883247 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 19:58:40.883247 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 19:58:40.883247 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 19:58:40.883247 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 19:58:40.887762 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 173/327] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 6 19:58:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 373 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:58:40.898267 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056a8256-1825-4ff1-bf84-36f577f322b1 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:41.098733 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:41.104246 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056a8256-1825-4ff1-bf84-36f577f322b1 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:41.192162 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04422ed-c2ad-4ca5-94b8-9ac6e7d22902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:41.193213 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04422ed-c2ad-4ca5-94b8-9ac6e7d22902 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:41.193755 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04422ed-c2ad-4ca5-94b8-9ac6e7d22902 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:41.194240 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04422ed-c2ad-4ca5-94b8-9ac6e7d22902 None admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:41.195535 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04422ed-c2ad-4ca5-94b8-9ac6e7d22902 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:58:41.196082 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04422ed-c2ad-4ca5-94b8-9ac6e7d22902 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:41.199064 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04422ed-c2ad-4ca5-94b8-9ac6e7d22902 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:41.220247 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 155/328] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Sep 6 19:58:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:41.229466 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834a88eb-e651-4366-a9e3-b2c26ee38c0d None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:41.443485 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834a88eb-e651-4366-a9e3-b2c26ee38c0d None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:41.494858 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04422ed-c2ad-4ca5-94b8-9ac6e7d22902 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:41.530002 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da3e312-46ac-44c6-b335-81962892dfd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:41.531649 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da3e312-46ac-44c6-b335-81962892dfd5 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:41.532156 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da3e312-46ac-44c6-b335-81962892dfd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:41.532671 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da3e312-46ac-44c6-b335-81962892dfd5 None admin] PATH_INFO: `/v3/users/8e81688c48174cefb1141af6f553ed82` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:41.537655 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da3e312-46ac-44c6-b335-81962892dfd5 None admin] RBAC: Authorizing `identity:get_user(user_id=8e81688c48174cefb1141af6f553ed82)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:41.540413 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da3e312-46ac-44c6-b335-81962892dfd5 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:41.545234 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 174/329] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 6 19:58:41 2022] GET /identity/v3/users/8e81688c48174cefb1141af6f553ed82 => generated 293 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:41.556253 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da3e312-46ac-44c6-b335-81962892dfd5 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:41.583956 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:41.686680 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04422ed-c2ad-4ca5-94b8-9ac6e7d22902 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:41.734804 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57361c2-24e4-4dcc-8a51-fcf044da026d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:41.735479 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57361c2-24e4-4dcc-8a51-fcf044da026d None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:41.735770 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57361c2-24e4-4dcc-8a51-fcf044da026d None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:41.736044 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57361c2-24e4-4dcc-8a51-fcf044da026d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:41.774755 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57361c2-24e4-4dcc-8a51-fcf044da026d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 19:58:41.790052 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-e57361c2-24e4-4dcc-8a51-fcf044da026d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 19:58:41.790052 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:58:41.790052 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:58:41.790052 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:58:41.790052 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:58:41.790052 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:58:41.790052 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:58:41.790052 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:58:41.790052 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:58:41.790052 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:58:41.790052 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:58:41.790052 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:58:41.790052 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 19:58:41.790052 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 19:58:41.790052 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 19:58:41.790052 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 19:58:41.793408 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 19:58:41.793408 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 19:58:41.793408 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 19:58:41.793408 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 19:58:41.793408 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:58:41.793408 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:58:41.793408 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:58:41.793408 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:58:41.793408 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:58:41.793408 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:58:41.793408 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:58:41.793408 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 19:58:41.793408 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:58:41.793408 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:58:41.793408 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:58:41.793408 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:58:41.797018 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:58:41.797018 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:58:41.797018 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:58:41.797018 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:58:41.797018 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 19:58:41.797018 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 19:58:41.797018 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 19:58:41.797018 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 19:58:41.797018 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 19:58:41.797018 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 19:58:41.797018 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 19:58:41.797018 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 19:58:41.797018 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 156/330] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 6 19:58:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 19:58:41.807429 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57361c2-24e4-4dcc-8a51-fcf044da026d None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:41.835529 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da3e312-46ac-44c6-b335-81962892dfd5 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:41.910179 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:41.910581 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4b4838-ff27-4364-bca0-0c394b2fa69e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:41.911345 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4b4838-ff27-4364-bca0-0c394b2fa69e None admin] REQUEST_METHOD: `PUT` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:41.912493 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4b4838-ff27-4364-bca0-0c394b2fa69e None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:41.912802 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4b4838-ff27-4364-bca0-0c394b2fa69e None admin] PATH_INFO: `/v3/system/users/8e81688c48174cefb1141af6f553ed82/roles/034691b44a684a7b8b9bf9f601dd64e9` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:41.915098 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4b4838-ff27-4364-bca0-0c394b2fa69e None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=8e81688c48174cefb1141af6f553ed82, role_id=034691b44a684a7b8b9bf9f601dd64e9)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:41.916660 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4b4838-ff27-4364-bca0-0c394b2fa69e None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:41.932943 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 175/331] 10.222.0.20 () {58 vars in 1334 bytes} [Tue Sep 6 19:58:41 2022] PUT /identity/v3/system/users/8e81688c48174cefb1141af6f553ed82/roles/034691b44a684a7b8b9bf9f601dd64e9 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 19:58:42.083519 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57361c2-24e4-4dcc-8a51-fcf044da026d None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:42.178908 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d4bc95-56f2-42a4-8835-395ec4e03ce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:42.180023 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d4bc95-56f2-42a4-8835-395ec4e03ce9 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:42.180383 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d4bc95-56f2-42a4-8835-395ec4e03ce9 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:42.180858 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d4bc95-56f2-42a4-8835-395ec4e03ce9 None admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:42.182116 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d4bc95-56f2-42a4-8835-395ec4e03ce9 None admin] RBAC: Adding query filter params (name=member) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:58:42.182658 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d4bc95-56f2-42a4-8835-395ec4e03ce9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:42.185199 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d4bc95-56f2-42a4-8835-395ec4e03ce9 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:42.206118 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 157/332] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Sep 6 19:58:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 400 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:42.215523 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4b4838-ff27-4364-bca0-0c394b2fa69e None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:42.354396 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:42.446505 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4b4838-ff27-4364-bca0-0c394b2fa69e None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:42.499292 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67467870-a3ce-4ae9-8aa9-dccd556fb3be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:42.500016 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67467870-a3ce-4ae9-8aa9-dccd556fb3be None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:42.500292 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67467870-a3ce-4ae9-8aa9-dccd556fb3be None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:42.500573 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67467870-a3ce-4ae9-8aa9-dccd556fb3be None admin] PATH_INFO: `/v3/users/c85315ab3f5e418dbcfc1f4071e3a53b` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:42.501911 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67467870-a3ce-4ae9-8aa9-dccd556fb3be None admin] RBAC: Authorizing `identity:get_user(user_id=c85315ab3f5e418dbcfc1f4071e3a53b)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:42.503560 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67467870-a3ce-4ae9-8aa9-dccd556fb3be None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:42.505342 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 176/333] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 6 19:58:42 2022] GET /identity/v3/users/c85315ab3f5e418dbcfc1f4071e3a53b => generated 293 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:42.512729 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d4bc95-56f2-42a4-8835-395ec4e03ce9 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:42.735156 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:42.768837 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d4bc95-56f2-42a4-8835-395ec4e03ce9 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:42.838044 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4597da78-9b05-4728-a2e7-5b9d48377d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:42.839144 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4597da78-9b05-4728-a2e7-5b9d48377d33 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:42.839642 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4597da78-9b05-4728-a2e7-5b9d48377d33 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:42.840135 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4597da78-9b05-4728-a2e7-5b9d48377d33 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:42.841588 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4597da78-9b05-4728-a2e7-5b9d48377d33 None admin] RBAC: Adding query filter params (role.id=4eea39e24b06431aa357053fe7907a3d, user.id=c85315ab3f5e418dbcfc1f4071e3a53b, scope.system=all) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:58:42.842170 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4597da78-9b05-4728-a2e7-5b9d48377d33 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:42.845105 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4597da78-9b05-4728-a2e7-5b9d48377d33 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:42.870350 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 158/334] 10.222.0.20 () {56 vars in 1308 bytes} [Tue Sep 6 19:58:42 2022] GET /identity/v3/role_assignments?user.id=c85315ab3f5e418dbcfc1f4071e3a53b&scope.system=all&role.id=4eea39e24b06431aa357053fe7907a3d => generated 510 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 19:58:45.123828 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa685ab-c89d-4dbc-843a-acd08c73bbb1 None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:45.125140 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa685ab-c89d-4dbc-843a-acd08c73bbb1 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:45.125140 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa685ab-c89d-4dbc-843a-acd08c73bbb1 None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:45.126902 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 177/335] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 19:58:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 19:58:45.136942 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba4b965-8deb-41db-b909-4672ac82c95a None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:45.137891 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba4b965-8deb-41db-b909-4672ac82c95a None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:45.138120 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba4b965-8deb-41db-b909-4672ac82c95a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:45.178198 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-3ba4b965-8deb-41db-b909-4672ac82c95a None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 19:58:45.180253 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-3ba4b965-8deb-41db-b909-4672ac82c95a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 19:58:45.345427 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 159/336] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 19:58:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 19:58:45.421135 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8242a6b8-0d51-4886-bcb5-e12a6b1217c2 None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:45.421135 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8242a6b8-0d51-4886-bcb5-e12a6b1217c2 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:45.421135 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8242a6b8-0d51-4886-bcb5-e12a6b1217c2 None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:45.421135 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 178/337] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 19:58:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 19:58:45.440528 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd06ecf6-c8bd-4cf4-9ca6-46cf2b58161a None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:45.441012 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd06ecf6-c8bd-4cf4-9ca6-46cf2b58161a None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:45.441479 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd06ecf6-c8bd-4cf4-9ca6-46cf2b58161a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:45.482964 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-bd06ecf6-c8bd-4cf4-9ca6-46cf2b58161a None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 19:58:45.485061 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-bd06ecf6-c8bd-4cf4-9ca6-46cf2b58161a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 19:58:45.666733 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf0313e-415c-4408-90d5-42401c9ed145 None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:45.667498 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf0313e-415c-4408-90d5-42401c9ed145 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:45.667986 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf0313e-415c-4408-90d5-42401c9ed145 None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:45.670480 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 179/338] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 19:58:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 19:58:45.680234 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26b883a-5366-4944-8949-550cc930f15c None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:45.680829 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26b883a-5366-4944-8949-550cc930f15c None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:45.681308 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26b883a-5366-4944-8949-550cc930f15c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:45.681412 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 160/339] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 19:58:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 19:58:45.716278 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-e26b883a-5366-4944-8949-550cc930f15c None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 19:58:45.718054 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-e26b883a-5366-4944-8949-550cc930f15c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 19:58:45.840079 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 180/340] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 19:58:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 19:58:45.970096 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad0decb-5967-415a-a916-e89640e9fd0c None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:45.971110 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad0decb-5967-415a-a916-e89640e9fd0c None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:45.971865 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad0decb-5967-415a-a916-e89640e9fd0c None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:45.974712 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 161/341] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 19:58:45 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 19:58:45.980879 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b404ac54-1d42-48eb-b3ae-5fe7af02b585 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:45.981162 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b404ac54-1d42-48eb-b3ae-5fe7af02b585 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:45.981603 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b404ac54-1d42-48eb-b3ae-5fe7af02b585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:46.014779 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-b404ac54-1d42-48eb-b3ae-5fe7af02b585 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 19:58:46.016417 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-b404ac54-1d42-48eb-b3ae-5fe7af02b585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 19:58:46.118775 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:46.125907 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 181/342] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 19:58:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 19:58:46.233377 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad0decb-5967-415a-a916-e89640e9fd0c None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:46.238422 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:46.479672 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad0decb-5967-415a-a916-e89640e9fd0c None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:46.564250 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b404ac54-1d42-48eb-b3ae-5fe7af02b585 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:46.569892 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba946bd-458d-4bb9-aff4-1997e6cea9eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:46.570983 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba946bd-458d-4bb9-aff4-1997e6cea9eb None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:46.571446 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba946bd-458d-4bb9-aff4-1997e6cea9eb None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:46.571907 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba946bd-458d-4bb9-aff4-1997e6cea9eb None admin] PATH_INFO: `/v3/roles/member` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:46.610629 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba946bd-458d-4bb9-aff4-1997e6cea9eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 19:58:46.621360 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-dba946bd-458d-4bb9-aff4-1997e6cea9eb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 19:58:46.621360 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:58:46.621360 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:58:46.621360 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:58:46.621360 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:58:46.621360 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:58:46.621360 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:58:46.621360 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:58:46.621360 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:58:46.621360 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:58:46.621360 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:58:46.621360 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:58:46.621360 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 19:58:46.621360 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 19:58:46.621360 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 19:58:46.621360 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 19:58:46.622383 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 19:58:46.622383 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 19:58:46.622383 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 19:58:46.622383 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 19:58:46.622383 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:58:46.622383 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:58:46.622383 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:58:46.622383 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:58:46.622383 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:58:46.622383 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:58:46.622383 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:58:46.622383 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 19:58:46.622383 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:58:46.622383 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:58:46.622383 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:58:46.622383 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:58:46.623166 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:58:46.623166 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:58:46.623166 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:58:46.623166 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:58:46.623166 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 19:58:46.623166 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 19:58:46.623166 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 19:58:46.623166 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 19:58:46.623166 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 19:58:46.623166 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 19:58:46.623166 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 19:58:46.623166 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 19:58:46.623784 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 162/343] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 6 19:58:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 391 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:58:46.625540 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba946bd-458d-4bb9-aff4-1997e6cea9eb None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:46.762281 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:46.834571 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba946bd-458d-4bb9-aff4-1997e6cea9eb None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:46.849103 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b404ac54-1d42-48eb-b3ae-5fe7af02b585 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:46.907231 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51292bd-252f-4525-9fb2-566b90dd07a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:46.908726 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51292bd-252f-4525-9fb2-566b90dd07a0 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:46.909454 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51292bd-252f-4525-9fb2-566b90dd07a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:46.910099 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51292bd-252f-4525-9fb2-566b90dd07a0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:46.919939 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95e8e5b-ce77-4af2-ac86-b6c70123c9af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:46.921086 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95e8e5b-ce77-4af2-ac86-b6c70123c9af None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:46.921745 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95e8e5b-ce77-4af2-ac86-b6c70123c9af None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:46.922248 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95e8e5b-ce77-4af2-ac86-b6c70123c9af None admin] PATH_INFO: `/v3/roles/member` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:46.931768 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51292bd-252f-4525-9fb2-566b90dd07a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 19:58:46.943412 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-b51292bd-252f-4525-9fb2-566b90dd07a0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 19:58:46.943412 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:58:46.943412 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:58:46.943412 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:58:46.943412 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:58:46.943412 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:58:46.943412 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:58:46.943412 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:58:46.943412 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:58:46.943412 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:58:46.943412 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:58:46.943412 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:58:46.943412 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 19:58:46.943412 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 19:58:46.943412 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 19:58:46.943412 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 19:58:46.945486 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 19:58:46.945486 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 19:58:46.945486 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 19:58:46.945486 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 19:58:46.945486 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:58:46.945486 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:58:46.945486 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:58:46.945486 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:58:46.945486 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:58:46.945486 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:58:46.945486 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:58:46.945486 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 19:58:46.945486 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:58:46.945486 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:58:46.945486 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:58:46.945486 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:58:46.947643 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:58:46.947643 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:58:46.947643 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:58:46.947643 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:58:46.947643 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 19:58:46.947643 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 19:58:46.947643 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 19:58:46.947643 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 19:58:46.947643 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 19:58:46.947643 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 19:58:46.947643 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 19:58:46.947643 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 19:58:46.949110 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95e8e5b-ce77-4af2-ac86-b6c70123c9af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 19:58:46.951767 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 182/344] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Sep 6 19:58:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 390 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:58:46.953530 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51292bd-252f-4525-9fb2-566b90dd07a0 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:46.958427 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-a95e8e5b-ce77-4af2-ac86-b6c70123c9af None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 19:58:46.958427 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:58:46.958427 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:58:46.958427 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:58:46.958427 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:58:46.958427 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:58:46.958427 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:58:46.958427 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:58:46.958427 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:58:46.958427 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:58:46.958427 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:58:46.958427 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:58:46.958427 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 19:58:46.958427 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 19:58:46.958427 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 19:58:46.958427 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 19:58:46.960731 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 19:58:46.960731 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 19:58:46.960731 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 19:58:46.960731 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 19:58:46.960731 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:58:46.960731 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:58:46.960731 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:58:46.960731 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:58:46.960731 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:58:46.960731 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:58:46.960731 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:58:46.960731 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 19:58:46.960731 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:58:46.960731 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:58:46.960731 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:58:46.960731 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:58:46.962604 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:58:46.962604 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:58:46.962604 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:58:46.962604 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:58:46.962604 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 19:58:46.962604 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 19:58:46.962604 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 19:58:46.962604 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 19:58:46.962604 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 19:58:46.962604 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 19:58:46.962604 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 19:58:46.962604 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 19:58:46.967773 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 163/345] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 6 19:58:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:58:46.967773 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b224a6d-7afd-4b53-866d-1259aa5ee388 None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:46.967773 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b224a6d-7afd-4b53-866d-1259aa5ee388 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:46.968842 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b224a6d-7afd-4b53-866d-1259aa5ee388 None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:46.974734 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 164/346] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 19:58:46 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 19:58:46.976591 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b224a6d-7afd-4b53-866d-1259aa5ee388 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:47.080976 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:47.162003 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51292bd-252f-4525-9fb2-566b90dd07a0 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:47.183179 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b224a6d-7afd-4b53-866d-1259aa5ee388 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:47.230284 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:47.237220 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153d1503-6daa-4f99-b01d-847dbb0dc633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:47.238330 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153d1503-6daa-4f99-b01d-847dbb0dc633 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:47.239054 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153d1503-6daa-4f99-b01d-847dbb0dc633 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:47.239623 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153d1503-6daa-4f99-b01d-847dbb0dc633 None admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:47.241205 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153d1503-6daa-4f99-b01d-847dbb0dc633 None admin] RBAC: Adding query filter params (name=member) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:58:47.242340 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153d1503-6daa-4f99-b01d-847dbb0dc633 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:47.244534 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153d1503-6daa-4f99-b01d-847dbb0dc633 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:47.256378 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbc3ef6-2789-4e6b-8142-4be4659e1f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:47.258092 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbc3ef6-2789-4e6b-8142-4be4659e1f79 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:47.258092 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbc3ef6-2789-4e6b-8142-4be4659e1f79 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:47.258344 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbc3ef6-2789-4e6b-8142-4be4659e1f79 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:47.266249 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 183/347] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Sep 6 19:58:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:47.267751 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153d1503-6daa-4f99-b01d-847dbb0dc633 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:47.281544 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbc3ef6-2789-4e6b-8142-4be4659e1f79 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 19:58:47.288344 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-cbbc3ef6-2789-4e6b-8142-4be4659e1f79 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 19:58:47.288344 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:58:47.288344 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:58:47.288344 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:58:47.288344 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:58:47.288344 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:58:47.288344 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:58:47.288344 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:58:47.288344 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:58:47.288344 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:58:47.288344 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:58:47.288344 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:58:47.288344 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 19:58:47.288344 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 19:58:47.288344 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 19:58:47.288344 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 19:58:47.290132 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 19:58:47.290132 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 19:58:47.290132 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 19:58:47.290132 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 19:58:47.290132 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:58:47.290132 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:58:47.290132 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:58:47.290132 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:58:47.290132 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:58:47.290132 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:58:47.290132 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:58:47.290132 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 19:58:47.290132 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:58:47.290132 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:58:47.290132 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:58:47.290132 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:58:47.291727 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:58:47.291727 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:58:47.291727 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:58:47.291727 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:58:47.291727 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 19:58:47.291727 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 19:58:47.291727 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 19:58:47.291727 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 19:58:47.291727 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 19:58:47.291727 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 19:58:47.291727 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 19:58:47.291727 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 19:58:47.291727 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 165/348] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Sep 6 19:58:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:58:47.291727 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbc3ef6-2789-4e6b-8142-4be4659e1f79 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:47.437554 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153d1503-6daa-4f99-b01d-847dbb0dc633 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:47.478468 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:47.503170 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbc3ef6-2789-4e6b-8142-4be4659e1f79 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:47.512378 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f407e6cf-4ff0-47f8-9978-864195c101f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:47.513443 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f407e6cf-4ff0-47f8-9978-864195c101f3 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:47.513968 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f407e6cf-4ff0-47f8-9978-864195c101f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:47.514454 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f407e6cf-4ff0-47f8-9978-864195c101f3 None admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:47.515845 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f407e6cf-4ff0-47f8-9978-864195c101f3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:58:47.516399 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f407e6cf-4ff0-47f8-9978-864195c101f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:47.519126 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f407e6cf-4ff0-47f8-9978-864195c101f3 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:47.539621 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 184/349] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 6 19:58:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:47.545687 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105d38c5-ec80-4073-9d48-9f52eefedee0 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:47.545687 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105d38c5-ec80-4073-9d48-9f52eefedee0 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:47.545687 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105d38c5-ec80-4073-9d48-9f52eefedee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:47.575906 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-105d38c5-ec80-4073-9d48-9f52eefedee0 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 19:58:47.577811 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-105d38c5-ec80-4073-9d48-9f52eefedee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 19:58:47.582172 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94656b6-0e89-454d-927b-49ff803f1b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:47.583127 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94656b6-0e89-454d-927b-49ff803f1b9d None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:47.583646 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94656b6-0e89-454d-927b-49ff803f1b9d None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:47.584083 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94656b6-0e89-454d-927b-49ff803f1b9d None admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:47.584945 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94656b6-0e89-454d-927b-49ff803f1b9d None admin] RBAC: Adding query filter params (name=member) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:58:47.585376 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94656b6-0e89-454d-927b-49ff803f1b9d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:47.588199 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94656b6-0e89-454d-927b-49ff803f1b9d None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:47.606246 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 166/350] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Sep 6 19:58:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:47.607839 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94656b6-0e89-454d-927b-49ff803f1b9d None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:47.698225 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:47.721723 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 185/351] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 19:58:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 19:58:47.723650 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105d38c5-ec80-4073-9d48-9f52eefedee0 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:47.804687 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94656b6-0e89-454d-927b-49ff803f1b9d None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:47.881207 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c88062-103e-430a-9cca-a7ed78d794bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:47.882250 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c88062-103e-430a-9cca-a7ed78d794bd None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:47.882732 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c88062-103e-430a-9cca-a7ed78d794bd None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:47.883133 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c88062-103e-430a-9cca-a7ed78d794bd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:47.934689 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105d38c5-ec80-4073-9d48-9f52eefedee0 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:47.964150 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:48.012486 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9621ed70-f0c7-4e7e-b251-3b8355686850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:48.013134 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9621ed70-f0c7-4e7e-b251-3b8355686850 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:48.013458 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9621ed70-f0c7-4e7e-b251-3b8355686850 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:48.013999 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9621ed70-f0c7-4e7e-b251-3b8355686850 None admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:48.014838 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9621ed70-f0c7-4e7e-b251-3b8355686850 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:58:48.015686 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9621ed70-f0c7-4e7e-b251-3b8355686850 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:48.018710 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9621ed70-f0c7-4e7e-b251-3b8355686850 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:48.035115 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 186/352] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 6 19:58:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:48.037901 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fa40b3-135c-4aae-bea9-82cc61873429 None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:48.038403 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fa40b3-135c-4aae-bea9-82cc61873429 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:48.038896 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fa40b3-135c-4aae-bea9-82cc61873429 None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:48.041926 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 187/353] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 19:58:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 19:58:48.043506 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fa40b3-135c-4aae-bea9-82cc61873429 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:48.051437 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c88062-103e-430a-9cca-a7ed78d794bd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:48.053946 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c88062-103e-430a-9cca-a7ed78d794bd None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:48.133503 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:48.223071 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fa40b3-135c-4aae-bea9-82cc61873429 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:48.233578 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 167/354] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 6 19:58:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 627 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 19:58:48.239199 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c88062-103e-430a-9cca-a7ed78d794bd None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:48.288034 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2972363-ce80-4077-ae34-8cf313d1e4ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:48.288906 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2972363-ce80-4077-ae34-8cf313d1e4ac None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:48.289326 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2972363-ce80-4077-ae34-8cf313d1e4ac None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:48.289843 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2972363-ce80-4077-ae34-8cf313d1e4ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:48.357603 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:48.402519 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2972363-ce80-4077-ae34-8cf313d1e4ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:48.404837 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2972363-ce80-4077-ae34-8cf313d1e4ac None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:48.432276 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c88062-103e-430a-9cca-a7ed78d794bd None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:48.511833 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af5674d-eab5-4be6-b021-57abb9c78a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:48.512458 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af5674d-eab5-4be6-b021-57abb9c78a0e None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:48.512742 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af5674d-eab5-4be6-b021-57abb9c78a0e None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:48.513801 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af5674d-eab5-4be6-b021-57abb9c78a0e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:48.535210 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:48.616845 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 188/355] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 6 19:58:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 574 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 19:58:48.618427 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2972363-ce80-4077-ae34-8cf313d1e4ac None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:48.652900 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af5674d-eab5-4be6-b021-57abb9c78a0e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:48.655750 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af5674d-eab5-4be6-b021-57abb9c78a0e None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:48.744156 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:48.780349 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2972363-ce80-4077-ae34-8cf313d1e4ac None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:48.804276 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 168/356] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 6 19:58:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 570 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 19:58:48.807498 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f969f665-fe22-4acb-bae4-4947b90d13ec None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:48.807966 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f969f665-fe22-4acb-bae4-4947b90d13ec None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:48.808426 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f969f665-fe22-4acb-bae4-4947b90d13ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:48.830984 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-f969f665-fe22-4acb-bae4-4947b90d13ec None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 19:58:48.832251 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-f969f665-fe22-4acb-bae4-4947b90d13ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 19:58:48.851040 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c1c837-7520-4a55-8f6c-e05743b4ea21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:48.851695 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c1c837-7520-4a55-8f6c-e05743b4ea21 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:48.852516 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c1c837-7520-4a55-8f6c-e05743b4ea21 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:48.852516 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c1c837-7520-4a55-8f6c-e05743b4ea21 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:48.869219 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c1c837-7520-4a55-8f6c-e05743b4ea21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 19:58:48.878390 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-f1c1c837-7520-4a55-8f6c-e05743b4ea21 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 19:58:48.878390 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:58:48.878390 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:58:48.878390 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:58:48.878390 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:58:48.878390 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:58:48.878390 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:58:48.878390 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:58:48.878390 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:58:48.878390 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:58:48.878390 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:58:48.878390 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:58:48.878390 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 19:58:48.878390 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 19:58:48.878390 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 19:58:48.878390 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 19:58:48.880334 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 19:58:48.880334 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 19:58:48.880334 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 19:58:48.880334 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 19:58:48.880334 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:58:48.880334 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:58:48.880334 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:58:48.880334 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:58:48.880334 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:58:48.880334 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:58:48.880334 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:58:48.880334 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 19:58:48.880334 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:58:48.880334 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:58:48.880334 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:58:48.880334 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:58:48.882244 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:58:48.882244 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:58:48.882244 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:58:48.882244 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:58:48.882244 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 19:58:48.882244 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 19:58:48.882244 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 19:58:48.882244 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 19:58:48.882244 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 19:58:48.882244 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 19:58:48.882244 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 19:58:48.882244 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 19:58:48.882244 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 189/357] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 6 19:58:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:58:48.882244 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c1c837-7520-4a55-8f6c-e05743b4ea21 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:48.894704 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:48.941024 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 169/358] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 19:58:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 19:58:48.942131 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f969f665-fe22-4acb-bae4-4947b90d13ec None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:49.010449 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c1c837-7520-4a55-8f6c-e05743b4ea21 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:49.058635 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a8770c-3ff5-4da0-9ccf-4270b6f461e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:49.059978 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a8770c-3ff5-4da0-9ccf-4270b6f461e6 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:49.060473 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a8770c-3ff5-4da0-9ccf-4270b6f461e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:49.061119 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a8770c-3ff5-4da0-9ccf-4270b6f461e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:49.066384 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:49.102779 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f969f665-fe22-4acb-bae4-4947b90d13ec None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:49.151919 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cc2eec-1db7-41e5-b65c-8e58b543c184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:49.152846 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cc2eec-1db7-41e5-b65c-8e58b543c184 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:49.153324 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cc2eec-1db7-41e5-b65c-8e58b543c184 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:49.153989 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cc2eec-1db7-41e5-b65c-8e58b543c184 None admin] PATH_INFO: `/v3/projects/22a00d6dfd004be4aa2f9727f02e69ab` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:49.155630 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cc2eec-1db7-41e5-b65c-8e58b543c184 None admin] RBAC: Authorizing `identity:get_project(project_id=22a00d6dfd004be4aa2f9727f02e69ab)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:49.158001 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cc2eec-1db7-41e5-b65c-8e58b543c184 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:49.160148 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 170/359] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 6 19:58:48 2022] GET /identity/v3/projects/22a00d6dfd004be4aa2f9727f02e69ab => generated 302 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:49.161634 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cc2eec-1db7-41e5-b65c-8e58b543c184 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:49.166848 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a8770c-3ff5-4da0-9ccf-4270b6f461e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:49.169428 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a8770c-3ff5-4da0-9ccf-4270b6f461e6 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:49.237128 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:49.320494 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 190/360] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 6 19:58:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 19:58:49.321379 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a8770c-3ff5-4da0-9ccf-4270b6f461e6 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:49.323055 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cc2eec-1db7-41e5-b65c-8e58b543c184 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:49.370385 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:49.375194 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e896d3ff-cecc-4d70-8d60-a4ac585f5f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:49.375915 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e896d3ff-cecc-4d70-8d60-a4ac585f5f7d None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:49.376281 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e896d3ff-cecc-4d70-8d60-a4ac585f5f7d None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:49.376573 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e896d3ff-cecc-4d70-8d60-a4ac585f5f7d None admin] PATH_INFO: `/v3/projects/197e55aa210c42abbf12e585a968f8c5` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:49.378089 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e896d3ff-cecc-4d70-8d60-a4ac585f5f7d None admin] RBAC: Authorizing `identity:get_project(project_id=197e55aa210c42abbf12e585a968f8c5)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:49.379579 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e896d3ff-cecc-4d70-8d60-a4ac585f5f7d None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:49.381338 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 171/361] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 6 19:58:49 2022] GET /identity/v3/projects/197e55aa210c42abbf12e585a968f8c5 => generated 298 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:49.382174 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e896d3ff-cecc-4d70-8d60-a4ac585f5f7d None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:49.458499 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a8770c-3ff5-4da0-9ccf-4270b6f461e6 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:49.504538 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cb89c0-e1ac-435b-b352-1496f3df78a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:49.505363 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cb89c0-e1ac-435b-b352-1496f3df78a6 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:49.505738 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cb89c0-e1ac-435b-b352-1496f3df78a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:49.506089 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cb89c0-e1ac-435b-b352-1496f3df78a6 None admin] PATH_INFO: `/v3/projects/197e55aa210c42abbf12e585a968f8c5` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:49.507840 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cb89c0-e1ac-435b-b352-1496f3df78a6 None admin] RBAC: Authorizing `identity:get_project(project_id=197e55aa210c42abbf12e585a968f8c5)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:49.509901 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cb89c0-e1ac-435b-b352-1496f3df78a6 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:49.511142 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e896d3ff-cecc-4d70-8d60-a4ac585f5f7d None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:49.512180 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 191/362] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 6 19:58:49 2022] GET /identity/v3/projects/197e55aa210c42abbf12e585a968f8c5 => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:49.513359 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cb89c0-e1ac-435b-b352-1496f3df78a6 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:49.528317 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:49.565576 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f812d39-1762-45ce-a1b1-df847bb221b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:49.566286 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f812d39-1762-45ce-a1b1-df847bb221b4 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:49.566631 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f812d39-1762-45ce-a1b1-df847bb221b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:49.566838 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f812d39-1762-45ce-a1b1-df847bb221b4 None admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:49.567544 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f812d39-1762-45ce-a1b1-df847bb221b4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:58:49.567850 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f812d39-1762-45ce-a1b1-df847bb221b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:49.569284 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f812d39-1762-45ce-a1b1-df847bb221b4 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:49.579437 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 172/363] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Sep 6 19:58:49 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:49.580226 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f812d39-1762-45ce-a1b1-df847bb221b4 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:49.648867 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cb89c0-e1ac-435b-b352-1496f3df78a6 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:49.650898 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:49.696877 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e5e425-0648-486b-8947-4a252e242b20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:49.697602 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e5e425-0648-486b-8947-4a252e242b20 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:49.697920 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e5e425-0648-486b-8947-4a252e242b20 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:49.698202 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e5e425-0648-486b-8947-4a252e242b20 None admin] PATH_INFO: `/v3/groups/6a12d79ef6944c129aeb6eccf1643416` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:49.706263 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e5e425-0648-486b-8947-4a252e242b20 None admin] RBAC: Authorizing `identity:get_group(group_id=6a12d79ef6944c129aeb6eccf1643416)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:49.707675 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f812d39-1762-45ce-a1b1-df847bb221b4 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:49.707846 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e5e425-0648-486b-8947-4a252e242b20 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:49.709478 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 192/364] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Sep 6 19:58:49 2022] GET /identity/v3/groups/6a12d79ef6944c129aeb6eccf1643416 => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:49.709478 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e5e425-0648-486b-8947-4a252e242b20 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:49.755407 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff181b77-11e2-4534-a6ef-accc5af9c5f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:49.756085 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff181b77-11e2-4534-a6ef-accc5af9c5f2 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:49.756369 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff181b77-11e2-4534-a6ef-accc5af9c5f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:49.756650 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff181b77-11e2-4534-a6ef-accc5af9c5f2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:49.771310 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff181b77-11e2-4534-a6ef-accc5af9c5f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 19:58:49.777227 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-ff181b77-11e2-4534-a6ef-accc5af9c5f2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 19:58:49.777227 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:58:49.777227 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:58:49.777227 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:58:49.777227 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:58:49.777227 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:58:49.777227 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:58:49.777227 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:58:49.777227 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:58:49.777227 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:58:49.777227 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:58:49.777227 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:58:49.777227 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 19:58:49.777227 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 19:58:49.777227 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 19:58:49.777227 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 19:58:49.778592 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 19:58:49.778592 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 19:58:49.778592 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 19:58:49.778592 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 19:58:49.778592 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:58:49.778592 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:58:49.778592 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:58:49.778592 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:58:49.778592 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:58:49.778592 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:58:49.778592 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:58:49.778592 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 19:58:49.778592 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:58:49.778592 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:58:49.778592 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:58:49.778592 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:58:49.779407 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:58:49.779407 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:58:49.779407 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:58:49.779407 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:58:49.779407 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 19:58:49.779407 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 19:58:49.779407 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 19:58:49.779407 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 19:58:49.779407 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 19:58:49.779407 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 19:58:49.779407 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 19:58:49.779407 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 19:58:49.779407 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 173/365] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Sep 6 19:58:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:58:49.780067 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff181b77-11e2-4534-a6ef-accc5af9c5f2 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:49.822729 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:49.852136 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e5e425-0648-486b-8947-4a252e242b20 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:49.897886 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1219d8a7-ebb0-4eff-8ddc-6cbb1addd029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:49.898791 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1219d8a7-ebb0-4eff-8ddc-6cbb1addd029 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:49.899250 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1219d8a7-ebb0-4eff-8ddc-6cbb1addd029 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:49.899631 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1219d8a7-ebb0-4eff-8ddc-6cbb1addd029 None admin] PATH_INFO: `/v3/projects/22a00d6dfd004be4aa2f9727f02e69ab` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:49.901808 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1219d8a7-ebb0-4eff-8ddc-6cbb1addd029 None admin] RBAC: Authorizing `identity:get_project(project_id=22a00d6dfd004be4aa2f9727f02e69ab)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:49.903684 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1219d8a7-ebb0-4eff-8ddc-6cbb1addd029 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:49.913751 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 193/366] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 6 19:58:49 2022] GET /identity/v3/projects/22a00d6dfd004be4aa2f9727f02e69ab => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:49.913751 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1219d8a7-ebb0-4eff-8ddc-6cbb1addd029 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:49.917668 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff181b77-11e2-4534-a6ef-accc5af9c5f2 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:49.956983 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:49.964458 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ea2010-f560-4c05-8f40-cc382e35cdc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:49.965189 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ea2010-f560-4c05-8f40-cc382e35cdc6 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:49.965664 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ea2010-f560-4c05-8f40-cc382e35cdc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:49.965986 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ea2010-f560-4c05-8f40-cc382e35cdc6 None admin] PATH_INFO: `/v3/groups/6a12d79ef6944c129aeb6eccf1643416` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:49.967247 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ea2010-f560-4c05-8f40-cc382e35cdc6 None admin] RBAC: Authorizing `identity:get_group(group_id=6a12d79ef6944c129aeb6eccf1643416)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:49.969808 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ea2010-f560-4c05-8f40-cc382e35cdc6 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:49.971818 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 174/367] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Sep 6 19:58:49 2022] GET /identity/v3/groups/6a12d79ef6944c129aeb6eccf1643416 => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:49.973202 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ea2010-f560-4c05-8f40-cc382e35cdc6 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:50.049525 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1219d8a7-ebb0-4eff-8ddc-6cbb1addd029 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:50.089591 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88e909c-7fc1-4277-8bcd-3c58398bcdbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:50.090365 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88e909c-7fc1-4277-8bcd-3c58398bcdbd None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:50.090863 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88e909c-7fc1-4277-8bcd-3c58398bcdbd None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:50.091379 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88e909c-7fc1-4277-8bcd-3c58398bcdbd None admin] PATH_INFO: `/v3/groups/6a12d79ef6944c129aeb6eccf1643416` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:50.092850 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88e909c-7fc1-4277-8bcd-3c58398bcdbd None admin] RBAC: Authorizing `identity:get_group(group_id=6a12d79ef6944c129aeb6eccf1643416)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:50.094626 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88e909c-7fc1-4277-8bcd-3c58398bcdbd None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:50.096387 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 194/368] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Sep 6 19:58:49 2022] GET /identity/v3/groups/6a12d79ef6944c129aeb6eccf1643416 => generated 226 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:50.097583 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88e909c-7fc1-4277-8bcd-3c58398bcdbd None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:50.105441 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:50.109492 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ea2010-f560-4c05-8f40-cc382e35cdc6 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:50.161403 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50bf852-c4de-4e5b-bb36-9bdb58e7da19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:50.162527 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50bf852-c4de-4e5b-bb36-9bdb58e7da19 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:50.163043 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50bf852-c4de-4e5b-bb36-9bdb58e7da19 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:50.163540 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50bf852-c4de-4e5b-bb36-9bdb58e7da19 None admin] PATH_INFO: `/v3/users/8e81688c48174cefb1141af6f553ed82` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:50.165998 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50bf852-c4de-4e5b-bb36-9bdb58e7da19 None admin] RBAC: Authorizing `identity:get_user(user_id=8e81688c48174cefb1141af6f553ed82)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:50.168782 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50bf852-c4de-4e5b-bb36-9bdb58e7da19 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:50.172443 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 175/369] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 6 19:58:49 2022] GET /identity/v3/users/8e81688c48174cefb1141af6f553ed82 => generated 293 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:50.173965 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50bf852-c4de-4e5b-bb36-9bdb58e7da19 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:50.236391 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88e909c-7fc1-4277-8bcd-3c58398bcdbd None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:50.242119 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:50.280446 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77f5d8a-328a-4a23-8f22-502586b91df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:50.281138 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77f5d8a-328a-4a23-8f22-502586b91df1 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:50.281431 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77f5d8a-328a-4a23-8f22-502586b91df1 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:50.281746 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77f5d8a-328a-4a23-8f22-502586b91df1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:50.282624 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77f5d8a-328a-4a23-8f22-502586b91df1 None admin] RBAC: Adding query filter params (group.id=6a12d79ef6944c129aeb6eccf1643416, role.id=4eea39e24b06431aa357053fe7907a3d, scope.project.id=22a00d6dfd004be4aa2f9727f02e69ab) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:58:50.283005 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77f5d8a-328a-4a23-8f22-502586b91df1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:50.284507 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77f5d8a-328a-4a23-8f22-502586b91df1 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:50.291153 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 195/370] 10.222.0.20 () {56 vars in 1376 bytes} [Tue Sep 6 19:58:50 2022] GET /identity/v3/role_assignments?group.id=6a12d79ef6944c129aeb6eccf1643416&scope.project.id=22a00d6dfd004be4aa2f9727f02e69ab&role.id=4eea39e24b06431aa357053fe7907a3d => generated 260 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:50.292029 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77f5d8a-328a-4a23-8f22-502586b91df1 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:50.321203 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50bf852-c4de-4e5b-bb36-9bdb58e7da19 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:50.375837 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7f378f-8fcd-499c-b180-1362eded9fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:50.376379 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7f378f-8fcd-499c-b180-1362eded9fab None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:50.376755 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7f378f-8fcd-499c-b180-1362eded9fab None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:50.377039 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7f378f-8fcd-499c-b180-1362eded9fab None admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:50.377893 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7f378f-8fcd-499c-b180-1362eded9fab None admin] RBAC: Adding query filter params (name=admin) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:58:50.378262 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7f378f-8fcd-499c-b180-1362eded9fab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:50.379820 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7f378f-8fcd-499c-b180-1362eded9fab None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:50.390970 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 176/371] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 6 19:58:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:50.391873 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7f378f-8fcd-499c-b180-1362eded9fab None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:50.442591 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:50.454877 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77f5d8a-328a-4a23-8f22-502586b91df1 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:50.508201 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76308501-7ebd-441a-b761-d09c294473fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:50.509315 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76308501-7ebd-441a-b761-d09c294473fa None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:50.509868 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76308501-7ebd-441a-b761-d09c294473fa None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:50.510368 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76308501-7ebd-441a-b761-d09c294473fa None admin] PATH_INFO: `/v3/groups/6a12d79ef6944c129aeb6eccf1643416` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:50.512739 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76308501-7ebd-441a-b761-d09c294473fa None admin] RBAC: Authorizing `identity:get_group(group_id=6a12d79ef6944c129aeb6eccf1643416)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:50.515750 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76308501-7ebd-441a-b761-d09c294473fa None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:50.518551 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 196/372] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Sep 6 19:58:50 2022] GET /identity/v3/groups/6a12d79ef6944c129aeb6eccf1643416 => generated 226 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:50.519711 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76308501-7ebd-441a-b761-d09c294473fa None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:50.588100 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7f378f-8fcd-499c-b180-1362eded9fab None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:50.588922 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:50.659153 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f952bc3e-2f56-4ec6-aeb4-6a0bd0179a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:50.660050 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f952bc3e-2f56-4ec6-aeb4-6a0bd0179a68 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:50.660480 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f952bc3e-2f56-4ec6-aeb4-6a0bd0179a68 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:50.660953 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f952bc3e-2f56-4ec6-aeb4-6a0bd0179a68 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:50.662411 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f952bc3e-2f56-4ec6-aeb4-6a0bd0179a68 None admin] RBAC: Adding query filter params (group.id=6a12d79ef6944c129aeb6eccf1643416, role.id=b02170460f054c259e825ab6e35f8537, scope.project.id=197e55aa210c42abbf12e585a968f8c5) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:58:50.662934 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f952bc3e-2f56-4ec6-aeb4-6a0bd0179a68 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:50.665639 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f952bc3e-2f56-4ec6-aeb4-6a0bd0179a68 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:50.676467 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 177/373] 10.222.0.20 () {56 vars in 1376 bytes} [Tue Sep 6 19:58:50 2022] GET /identity/v3/role_assignments?group.id=6a12d79ef6944c129aeb6eccf1643416&scope.project.id=197e55aa210c42abbf12e585a968f8c5&role.id=b02170460f054c259e825ab6e35f8537 => generated 260 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:50.678100 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f952bc3e-2f56-4ec6-aeb4-6a0bd0179a68 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:50.737580 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76308501-7ebd-441a-b761-d09c294473fa None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:50.808362 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b321da1d-4275-4219-ac35-26914cc76c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:50.810084 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b321da1d-4275-4219-ac35-26914cc76c30 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:50.810955 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b321da1d-4275-4219-ac35-26914cc76c30 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:50.811731 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b321da1d-4275-4219-ac35-26914cc76c30 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:50.813447 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b321da1d-4275-4219-ac35-26914cc76c30 None admin] RBAC: Adding query filter params (group.id=6a12d79ef6944c129aeb6eccf1643416, role.id=4eea39e24b06431aa357053fe7907a3d, scope.project.id=197e55aa210c42abbf12e585a968f8c5) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:58:50.814381 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b321da1d-4275-4219-ac35-26914cc76c30 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:50.817547 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b321da1d-4275-4219-ac35-26914cc76c30 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:50.833600 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 197/374] 10.222.0.20 () {56 vars in 1376 bytes} [Tue Sep 6 19:58:50 2022] GET /identity/v3/role_assignments?group.id=6a12d79ef6944c129aeb6eccf1643416&scope.project.id=197e55aa210c42abbf12e585a968f8c5&role.id=4eea39e24b06431aa357053fe7907a3d => generated 260 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:50.835378 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b321da1d-4275-4219-ac35-26914cc76c30 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:50.890764 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:50.914262 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f952bc3e-2f56-4ec6-aeb4-6a0bd0179a68 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:51.027446 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60faf02-da07-420a-83b7-41c7dc1ccf7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:51.028584 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60faf02-da07-420a-83b7-41c7dc1ccf7a None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:51.029284 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60faf02-da07-420a-83b7-41c7dc1ccf7a None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:51.029799 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60faf02-da07-420a-83b7-41c7dc1ccf7a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:51.031524 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60faf02-da07-420a-83b7-41c7dc1ccf7a None admin] RBAC: Adding query filter params (role.id=034691b44a684a7b8b9bf9f601dd64e9, user.id=8e81688c48174cefb1141af6f553ed82, scope.system=all) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:58:51.032275 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60faf02-da07-420a-83b7-41c7dc1ccf7a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:51.036869 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60faf02-da07-420a-83b7-41c7dc1ccf7a None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:51.063983 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 178/375] 10.222.0.20 () {56 vars in 1308 bytes} [Tue Sep 6 19:58:50 2022] GET /identity/v3/role_assignments?user.id=8e81688c48174cefb1141af6f553ed82&scope.system=all&role.id=034691b44a684a7b8b9bf9f601dd64e9 => generated 510 bytes in 387 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:51.065976 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60faf02-da07-420a-83b7-41c7dc1ccf7a None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:51.204202 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:51.215441 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b321da1d-4275-4219-ac35-26914cc76c30 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:51.291522 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27900bd-0b8f-439c-bd75-8e18a0533ef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:51.292573 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27900bd-0b8f-439c-bd75-8e18a0533ef6 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:51.293050 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27900bd-0b8f-439c-bd75-8e18a0533ef6 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:51.293530 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27900bd-0b8f-439c-bd75-8e18a0533ef6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:51.316923 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60faf02-da07-420a-83b7-41c7dc1ccf7a None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:51.433110 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7626d0a2-40fb-476e-ae9a-6f411988c3c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:51.434078 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7626d0a2-40fb-476e-ae9a-6f411988c3c6 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:51.434557 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7626d0a2-40fb-476e-ae9a-6f411988c3c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:51.435029 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7626d0a2-40fb-476e-ae9a-6f411988c3c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:51.436330 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7626d0a2-40fb-476e-ae9a-6f411988c3c6 None admin] RBAC: Adding query filter params (group.id=6a12d79ef6944c129aeb6eccf1643416, role.id=b02170460f054c259e825ab6e35f8537, scope.project.id=22a00d6dfd004be4aa2f9727f02e69ab) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:58:51.436850 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7626d0a2-40fb-476e-ae9a-6f411988c3c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:51.440244 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7626d0a2-40fb-476e-ae9a-6f411988c3c6 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:51.458640 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 179/376] 10.222.0.20 () {56 vars in 1376 bytes} [Tue Sep 6 19:58:51 2022] GET /identity/v3/role_assignments?group.id=6a12d79ef6944c129aeb6eccf1643416&scope.project.id=22a00d6dfd004be4aa2f9727f02e69ab&role.id=b02170460f054c259e825ab6e35f8537 => generated 260 bytes in 394 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:51.536566 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27900bd-0b8f-439c-bd75-8e18a0533ef6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:51.539231 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27900bd-0b8f-439c-bd75-8e18a0533ef6 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:51.764680 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:51.774287 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 198/377] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 6 19:58:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 940 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 19:58:51.784107 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7626d0a2-40fb-476e-ae9a-6f411988c3c6 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:51.830407 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:52.030125 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7626d0a2-40fb-476e-ae9a-6f411988c3c6 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:52.130522 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2ce0d3-e5a8-4d57-adbd-e744560544f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:52.131660 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2ce0d3-e5a8-4d57-adbd-e744560544f8 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:52.132177 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2ce0d3-e5a8-4d57-adbd-e744560544f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:52.132663 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2ce0d3-e5a8-4d57-adbd-e744560544f8 None admin] PATH_INFO: `/v3/projects/a630b7748ecd4893b8db1b6c1dc4efe9` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:52.134959 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2ce0d3-e5a8-4d57-adbd-e744560544f8 None admin] RBAC: Authorizing `identity:get_project(project_id=a630b7748ecd4893b8db1b6c1dc4efe9)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:52.138045 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2ce0d3-e5a8-4d57-adbd-e744560544f8 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:52.141534 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 180/378] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 6 19:58:51 2022] GET /identity/v3/projects/a630b7748ecd4893b8db1b6c1dc4efe9 => generated 344 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:52.148969 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27900bd-0b8f-439c-bd75-8e18a0533ef6 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:52.325136 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27900bd-0b8f-439c-bd75-8e18a0533ef6 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:52.366148 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a44336a-904f-4636-8ee9-0f56fb81a6f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:52.366825 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a44336a-904f-4636-8ee9-0f56fb81a6f7 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:52.367119 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a44336a-904f-4636-8ee9-0f56fb81a6f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:52.367404 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a44336a-904f-4636-8ee9-0f56fb81a6f7 None admin] PATH_INFO: `/v3/groups/c2db17bd65f74cfb8e12147856fca032` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:52.378122 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a44336a-904f-4636-8ee9-0f56fb81a6f7 None admin] RBAC: Authorizing `identity:get_group(group_id=c2db17bd65f74cfb8e12147856fca032)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:52.379792 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a44336a-904f-4636-8ee9-0f56fb81a6f7 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:52.382210 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 199/379] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Sep 6 19:58:52 2022] GET /identity/v3/groups/c2db17bd65f74cfb8e12147856fca032 => generated 229 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:52.389303 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2ce0d3-e5a8-4d57-adbd-e744560544f8 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:52.587537 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:52.617160 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2ce0d3-e5a8-4d57-adbd-e744560544f8 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:52.678496 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3177948-d2a7-4199-883c-7d6f82fb6a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:52.679126 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3177948-d2a7-4199-883c-7d6f82fb6a1b None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:52.679405 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3177948-d2a7-4199-883c-7d6f82fb6a1b None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:52.679679 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3177948-d2a7-4199-883c-7d6f82fb6a1b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:52.680605 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3177948-d2a7-4199-883c-7d6f82fb6a1b None admin] RBAC: Adding query filter params (group.id=c2db17bd65f74cfb8e12147856fca032, role.id=e0ab41d8ae22474688b5361bcc75e0f0, scope.project.id=a630b7748ecd4893b8db1b6c1dc4efe9) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:58:52.680919 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3177948-d2a7-4199-883c-7d6f82fb6a1b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:52.682552 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3177948-d2a7-4199-883c-7d6f82fb6a1b None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:52.694731 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 181/380] 10.222.0.20 () {56 vars in 1376 bytes} [Tue Sep 6 19:58:52 2022] GET /identity/v3/role_assignments?group.id=c2db17bd65f74cfb8e12147856fca032&scope.project.id=a630b7748ecd4893b8db1b6c1dc4efe9&role.id=e0ab41d8ae22474688b5361bcc75e0f0 => generated 260 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:53.608148 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40b0f67-92c7-43f4-86fc-e2af56bab37e None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:53.608992 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40b0f67-92c7-43f4-86fc-e2af56bab37e None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:53.608992 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40b0f67-92c7-43f4-86fc-e2af56bab37e None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:53.610596 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 200/381] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 19:58:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 19:58:53.616418 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeff54e-5ee0-45bd-bcde-816624f40e4f None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:53.617219 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeff54e-5ee0-45bd-bcde-816624f40e4f None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:53.617219 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeff54e-5ee0-45bd-bcde-816624f40e4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:53.638849 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-beeff54e-5ee0-45bd-bcde-816624f40e4f None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 19:58:53.644006 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-beeff54e-5ee0-45bd-bcde-816624f40e4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 19:58:53.727419 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 182/382] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 19:58:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 19:58:54.034457 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40b0f67-92c7-43f4-86fc-e2af56bab37e None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:54.130420 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:54.191333 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40b0f67-92c7-43f4-86fc-e2af56bab37e None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:54.235522 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f110fa5-f877-491b-bb8c-08a50ad149ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:54.236356 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f110fa5-f877-491b-bb8c-08a50ad149ea None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:54.236767 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f110fa5-f877-491b-bb8c-08a50ad149ea None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:54.237048 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f110fa5-f877-491b-bb8c-08a50ad149ea None admin] PATH_INFO: `/v3/roles/member` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:54.250516 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f110fa5-f877-491b-bb8c-08a50ad149ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 19:58:54.256044 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-6f110fa5-f877-491b-bb8c-08a50ad149ea None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 19:58:54.256044 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:58:54.256044 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:58:54.256044 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:58:54.256044 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:58:54.256044 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:58:54.256044 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:58:54.256044 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:58:54.256044 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:58:54.256044 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:58:54.256044 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:58:54.256044 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:58:54.256044 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 19:58:54.256044 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 19:58:54.256044 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 19:58:54.256044 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 19:58:54.258292 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 19:58:54.258292 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 19:58:54.258292 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 19:58:54.258292 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 19:58:54.258292 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:58:54.258292 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:58:54.258292 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:58:54.258292 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:58:54.258292 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:58:54.258292 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:58:54.258292 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:58:54.258292 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 19:58:54.258292 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:58:54.258292 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:58:54.258292 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:58:54.258292 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:58:54.260119 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:58:54.260119 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:58:54.260119 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:58:54.260119 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:58:54.260119 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 19:58:54.260119 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 19:58:54.260119 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 19:58:54.260119 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 19:58:54.260119 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 19:58:54.260119 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 19:58:54.260119 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 19:58:54.260119 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 19:58:54.260119 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 201/383] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 6 19:58:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:58:54.265132 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeff54e-5ee0-45bd-bcde-816624f40e4f None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:54.450352 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:54.491210 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeff54e-5ee0-45bd-bcde-816624f40e4f None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:54.543851 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ee081e-f631-4069-939d-93306d809b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:54.544866 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ee081e-f631-4069-939d-93306d809b56 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:54.545344 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ee081e-f631-4069-939d-93306d809b56 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:54.545894 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ee081e-f631-4069-939d-93306d809b56 None admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:54.547337 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ee081e-f631-4069-939d-93306d809b56 None admin] RBAC: Adding query filter params (name=member) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:58:54.547848 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ee081e-f631-4069-939d-93306d809b56 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:54.549598 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ee081e-f631-4069-939d-93306d809b56 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:54.560863 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 183/384] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Sep 6 19:58:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 19:58:54.571274 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f110fa5-f877-491b-bb8c-08a50ad149ea None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:54.762584 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ef53af-c304-47ae-9d01-6ccb08dce864 None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:54.763331 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ef53af-c304-47ae-9d01-6ccb08dce864 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:54.763331 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ef53af-c304-47ae-9d01-6ccb08dce864 None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:54.766603 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 184/385] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 19:58:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 19:58:54.772982 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600caba7-1a6b-4471-988d-5f62c8ce2fc1 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:54.776183 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600caba7-1a6b-4471-988d-5f62c8ce2fc1 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:54.777259 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600caba7-1a6b-4471-988d-5f62c8ce2fc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:54.846939 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-600caba7-1a6b-4471-988d-5f62c8ce2fc1 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 19:58:54.850153 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-600caba7-1a6b-4471-988d-5f62c8ce2fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 19:58:54.895231 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f110fa5-f877-491b-bb8c-08a50ad149ea None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:54.964872 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:55.004744 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f11702b-f0a5-4e4d-ad2e-1b71534c4a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:55.006461 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f11702b-f0a5-4e4d-ad2e-1b71534c4a5f None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:55.007323 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f11702b-f0a5-4e4d-ad2e-1b71534c4a5f None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:55.008037 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f11702b-f0a5-4e4d-ad2e-1b71534c4a5f None admin] PATH_INFO: `/v3/groups/6a12d79ef6944c129aeb6eccf1643416` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:55.010684 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f11702b-f0a5-4e4d-ad2e-1b71534c4a5f None admin] RBAC: Authorizing `identity:get_group(group_id=6a12d79ef6944c129aeb6eccf1643416)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:55.014735 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f11702b-f0a5-4e4d-ad2e-1b71534c4a5f None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:55.019119 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 202/386] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Sep 6 19:58:54 2022] GET /identity/v3/groups/6a12d79ef6944c129aeb6eccf1643416 => generated 226 bytes in 448 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 19:58:55.022080 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cac660-667b-4bcc-b443-6a51efc298fa None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:55.023055 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cac660-667b-4bcc-b443-6a51efc298fa None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:55.023805 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cac660-667b-4bcc-b443-6a51efc298fa None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:55.027029 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 203/387] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 19:58:55 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 19:58:55.027556 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 185/388] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 19:58:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 19:58:55.029305 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cac660-667b-4bcc-b443-6a51efc298fa None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:55.052048 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2793f67-b27c-4baa-883c-ad311599836a None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:55.052328 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2793f67-b27c-4baa-883c-ad311599836a None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:55.052700 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2793f67-b27c-4baa-883c-ad311599836a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:55.098777 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-d2793f67-b27c-4baa-883c-ad311599836a None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 19:58:55.100020 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-d2793f67-b27c-4baa-883c-ad311599836a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 19:58:55.238722 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cac660-667b-4bcc-b443-6a51efc298fa None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:55.316803 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd85b18-d97e-4b37-9dca-c3bf7e49778b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:55.317882 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd85b18-d97e-4b37-9dca-c3bf7e49778b None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:55.318390 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd85b18-d97e-4b37-9dca-c3bf7e49778b None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:55.318883 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd85b18-d97e-4b37-9dca-c3bf7e49778b None admin] PATH_INFO: `/v3/projects/22a00d6dfd004be4aa2f9727f02e69ab` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:55.321112 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd85b18-d97e-4b37-9dca-c3bf7e49778b None admin] RBAC: Authorizing `identity:get_project(project_id=22a00d6dfd004be4aa2f9727f02e69ab)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:55.325432 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd85b18-d97e-4b37-9dca-c3bf7e49778b None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:55.329550 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 204/389] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 6 19:58:55 2022] GET /identity/v3/projects/22a00d6dfd004be4aa2f9727f02e69ab => generated 302 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:55.333326 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:55.337808 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd85b18-d97e-4b37-9dca-c3bf7e49778b None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:55.343504 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 186/390] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 19:58:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 19:58:55.446292 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2793f67-b27c-4baa-883c-ad311599836a None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:55.488223 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:55.592403 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd85b18-d97e-4b37-9dca-c3bf7e49778b None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:55.619639 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2793f67-b27c-4baa-883c-ad311599836a None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:55.678165 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363b6cc0-7752-4117-8422-9bc4a5e5cdfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:55.678952 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363b6cc0-7752-4117-8422-9bc4a5e5cdfc None admin] REQUEST_METHOD: `PUT` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:55.679236 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363b6cc0-7752-4117-8422-9bc4a5e5cdfc None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:55.679521 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363b6cc0-7752-4117-8422-9bc4a5e5cdfc None admin] PATH_INFO: `/v3/projects/22a00d6dfd004be4aa2f9727f02e69ab/groups/6a12d79ef6944c129aeb6eccf1643416/roles/4eea39e24b06431aa357053fe7907a3d` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:55.682117 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363b6cc0-7752-4117-8422-9bc4a5e5cdfc None admin] RBAC: Authorizing `identity:create_grant(project_id=22a00d6dfd004be4aa2f9727f02e69ab, group_id=6a12d79ef6944c129aeb6eccf1643416, role_id=4eea39e24b06431aa357053fe7907a3d)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:55.682352 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbf06e2-c2ea-41d3-acab-3cea87ff2049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:55.682904 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbf06e2-c2ea-41d3-acab-3cea87ff2049 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:55.683182 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbf06e2-c2ea-41d3-acab-3cea87ff2049 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:55.683602 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbf06e2-c2ea-41d3-acab-3cea87ff2049 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:55.683771 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363b6cc0-7752-4117-8422-9bc4a5e5cdfc None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:55.699249 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbf06e2-c2ea-41d3-acab-3cea87ff2049 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 19:58:55.703285 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 205/391] 10.222.0.20 () {58 vars in 1442 bytes} [Tue Sep 6 19:58:55 2022] PUT /identity/v3/projects/22a00d6dfd004be4aa2f9727f02e69ab/groups/6a12d79ef6944c129aeb6eccf1643416/roles/4eea39e24b06431aa357053fe7907a3d => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 19:58:55.704558 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-4fbf06e2-c2ea-41d3-acab-3cea87ff2049 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 19:58:55.704558 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:58:55.704558 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:58:55.704558 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:58:55.704558 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:58:55.704558 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:58:55.704558 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:58:55.704558 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:58:55.704558 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:58:55.704558 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:58:55.704558 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:58:55.704558 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:58:55.704558 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 19:58:55.704558 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 19:58:55.704558 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 19:58:55.704558 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 19:58:55.714958 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 19:58:55.714958 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 19:58:55.714958 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 19:58:55.714958 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 19:58:55.714958 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:58:55.714958 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:58:55.714958 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:58:55.714958 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:58:55.714958 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:58:55.714958 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:58:55.714958 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:58:55.714958 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 19:58:55.714958 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:58:55.714958 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:58:55.714958 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:58:55.714958 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:58:55.716554 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:58:55.716554 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:58:55.716554 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:58:55.716554 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:58:55.716554 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 19:58:55.716554 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 19:58:55.716554 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 19:58:55.716554 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 19:58:55.716554 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 19:58:55.716554 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 19:58:55.716554 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 19:58:55.716554 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 19:58:55.716554 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 187/392] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 6 19:58:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:58:55.716554 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbf06e2-c2ea-41d3-acab-3cea87ff2049 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:55.719920 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306efd76-e683-4e2b-bff3-a9055994768c None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:55.719920 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306efd76-e683-4e2b-bff3-a9055994768c None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:55.719920 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306efd76-e683-4e2b-bff3-a9055994768c None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:55.719920 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 206/393] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 19:58:55 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 19:58:55.725565 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fd4d07-1f39-45f8-b457-878986f9ab4a None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:55.725565 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fd4d07-1f39-45f8-b457-878986f9ab4a None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:55.725565 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fd4d07-1f39-45f8-b457-878986f9ab4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:55.771115 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-15fd4d07-1f39-45f8-b457-878986f9ab4a None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 19:58:55.775242 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-15fd4d07-1f39-45f8-b457-878986f9ab4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 19:58:55.835336 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:55.909752 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 207/394] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 19:58:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 19:58:55.911781 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fd4d07-1f39-45f8-b457-878986f9ab4a None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:55.928891 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbf06e2-c2ea-41d3-acab-3cea87ff2049 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:55.986355 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd74fb7-9867-4782-93e5-68e2c8278a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:55.987030 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd74fb7-9867-4782-93e5-68e2c8278a6c None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:55.987371 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd74fb7-9867-4782-93e5-68e2c8278a6c None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:55.987642 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd74fb7-9867-4782-93e5-68e2c8278a6c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:56.002163 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd74fb7-9867-4782-93e5-68e2c8278a6c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 19:58:56.015896 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-0fd74fb7-9867-4782-93e5-68e2c8278a6c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 19:58:56.015896 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:58:56.015896 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:58:56.015896 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:58:56.015896 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:58:56.015896 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:58:56.015896 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:58:56.015896 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:58:56.015896 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:58:56.015896 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:58:56.015896 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:58:56.015896 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:58:56.015896 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 19:58:56.015896 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 19:58:56.015896 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 19:58:56.015896 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 19:58:56.017683 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 19:58:56.017683 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 19:58:56.017683 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 19:58:56.017683 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 19:58:56.017683 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:58:56.017683 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:58:56.017683 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:58:56.017683 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:58:56.017683 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:58:56.017683 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:58:56.017683 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:58:56.017683 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 19:58:56.017683 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:58:56.017683 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:58:56.017683 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:58:56.017683 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:58:56.019281 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:58:56.019281 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:58:56.019281 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:58:56.019281 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:58:56.019281 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 19:58:56.019281 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 19:58:56.019281 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 19:58:56.019281 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 19:58:56.019281 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 19:58:56.019281 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 19:58:56.019281 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 19:58:56.019281 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 19:58:56.019281 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 188/395] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Sep 6 19:58:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:58:56.019281 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd74fb7-9867-4782-93e5-68e2c8278a6c None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:56.030215 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:56.178282 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd74fb7-9867-4782-93e5-68e2c8278a6c None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:56.200919 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fd4d07-1f39-45f8-b457-878986f9ab4a None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:56.204782 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:56.250704 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a59d98-3f7c-4eee-8a6d-040aba3f78c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:56.251882 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a59d98-3f7c-4eee-8a6d-040aba3f78c3 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:56.252365 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a59d98-3f7c-4eee-8a6d-040aba3f78c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:56.252837 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a59d98-3f7c-4eee-8a6d-040aba3f78c3 None admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:56.254182 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a59d98-3f7c-4eee-8a6d-040aba3f78c3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:58:56.254721 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a59d98-3f7c-4eee-8a6d-040aba3f78c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:56.257383 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a59d98-3f7c-4eee-8a6d-040aba3f78c3 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:56.282589 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 189/396] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 6 19:58:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:56.284183 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a59d98-3f7c-4eee-8a6d-040aba3f78c3 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:56.314971 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91571d3d-f72d-49d3-873a-60a649d3fd04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:56.316061 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91571d3d-f72d-49d3-873a-60a649d3fd04 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:56.316547 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91571d3d-f72d-49d3-873a-60a649d3fd04 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:56.317843 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91571d3d-f72d-49d3-873a-60a649d3fd04 None admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:56.319558 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91571d3d-f72d-49d3-873a-60a649d3fd04 None admin] RBAC: Adding query filter params (name=member) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:58:56.320126 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91571d3d-f72d-49d3-873a-60a649d3fd04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:56.323486 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91571d3d-f72d-49d3-873a-60a649d3fd04 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:56.337445 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 208/397] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Sep 6 19:58:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 426 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:56.338720 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91571d3d-f72d-49d3-873a-60a649d3fd04 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:56.438736 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a59d98-3f7c-4eee-8a6d-040aba3f78c3 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:56.488873 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63cd887-35c0-45e2-a98d-c45ca53d4ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:56.489637 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63cd887-35c0-45e2-a98d-c45ca53d4ed8 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:56.489939 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63cd887-35c0-45e2-a98d-c45ca53d4ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:56.490221 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63cd887-35c0-45e2-a98d-c45ca53d4ed8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:56.491435 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91571d3d-f72d-49d3-873a-60a649d3fd04 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:56.505481 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63cd887-35c0-45e2-a98d-c45ca53d4ed8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 19:58:56.508639 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:56.513539 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-d63cd887-35c0-45e2-a98d-c45ca53d4ed8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 19:58:56.513539 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:58:56.513539 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:58:56.513539 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:58:56.513539 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:58:56.513539 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:58:56.513539 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:58:56.513539 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:58:56.513539 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:58:56.513539 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:58:56.513539 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:58:56.513539 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:58:56.513539 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 19:58:56.513539 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 19:58:56.513539 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 19:58:56.513539 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 19:58:56.514665 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 19:58:56.514665 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 19:58:56.514665 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 19:58:56.514665 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 19:58:56.514665 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:58:56.514665 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:58:56.514665 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:58:56.514665 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:58:56.514665 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:58:56.514665 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:58:56.514665 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:58:56.514665 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 19:58:56.514665 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:58:56.514665 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:58:56.514665 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:58:56.514665 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:58:56.516299 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:58:56.516299 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:58:56.516299 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:58:56.516299 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:58:56.516299 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 19:58:56.516299 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 19:58:56.516299 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 19:58:56.516299 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 19:58:56.516299 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 19:58:56.516299 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 19:58:56.516299 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 19:58:56.516299 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 19:58:56.520957 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 190/398] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Sep 6 19:58:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:58:56.521990 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63cd887-35c0-45e2-a98d-c45ca53d4ed8 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:56.552338 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31681d2-745e-41b4-96f4-4af0db427d10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:56.553386 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31681d2-745e-41b4-96f4-4af0db427d10 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:56.553928 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31681d2-745e-41b4-96f4-4af0db427d10 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:56.555180 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31681d2-745e-41b4-96f4-4af0db427d10 None admin] PATH_INFO: `/v3/groups/6a12d79ef6944c129aeb6eccf1643416` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:56.563587 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31681d2-745e-41b4-96f4-4af0db427d10 None admin] RBAC: Authorizing `identity:get_group(group_id=6a12d79ef6944c129aeb6eccf1643416)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:56.565440 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31681d2-745e-41b4-96f4-4af0db427d10 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:56.573499 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 209/399] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Sep 6 19:58:56 2022] GET /identity/v3/groups/6a12d79ef6944c129aeb6eccf1643416 => generated 226 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:56.573499 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31681d2-745e-41b4-96f4-4af0db427d10 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:56.613677 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:56.726127 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63cd887-35c0-45e2-a98d-c45ca53d4ed8 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:56.755327 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31681d2-745e-41b4-96f4-4af0db427d10 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:56.801553 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbd7fb8-8d03-4596-b5b7-ac0c8bbffb0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:56.802579 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbd7fb8-8d03-4596-b5b7-ac0c8bbffb0a None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:56.803028 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbd7fb8-8d03-4596-b5b7-ac0c8bbffb0a None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:56.803440 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbd7fb8-8d03-4596-b5b7-ac0c8bbffb0a None admin] PATH_INFO: `/v3/groups/6a12d79ef6944c129aeb6eccf1643416` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:56.804922 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbd7fb8-8d03-4596-b5b7-ac0c8bbffb0a None admin] RBAC: Authorizing `identity:get_group(group_id=6a12d79ef6944c129aeb6eccf1643416)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:56.806795 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbd7fb8-8d03-4596-b5b7-ac0c8bbffb0a None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:56.807262 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383f2535-36c2-483d-8418-9aaff44eda2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:56.808306 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383f2535-36c2-483d-8418-9aaff44eda2a None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:56.808794 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383f2535-36c2-483d-8418-9aaff44eda2a None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:56.809343 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 191/400] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Sep 6 19:58:56 2022] GET /identity/v3/groups/6a12d79ef6944c129aeb6eccf1643416 => generated 226 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:56.810378 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbd7fb8-8d03-4596-b5b7-ac0c8bbffb0a None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:56.816538 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383f2535-36c2-483d-8418-9aaff44eda2a None admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:56.818653 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383f2535-36c2-483d-8418-9aaff44eda2a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:58:56.818944 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383f2535-36c2-483d-8418-9aaff44eda2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:56.822006 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383f2535-36c2-483d-8418-9aaff44eda2a None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:56.839090 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 210/401] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 6 19:58:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:56.841343 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c148768b-6dbf-4a1a-ae18-7ed5861c0b9c None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:56.841889 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c148768b-6dbf-4a1a-ae18-7ed5861c0b9c None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:56.842382 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c148768b-6dbf-4a1a-ae18-7ed5861c0b9c None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:56.845767 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 211/402] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 19:58:56 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 19:58:56.847332 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c148768b-6dbf-4a1a-ae18-7ed5861c0b9c None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:56.928098 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:57.011209 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbd7fb8-8d03-4596-b5b7-ac0c8bbffb0a None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:57.027323 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:57.043854 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c148768b-6dbf-4a1a-ae18-7ed5861c0b9c None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:57.060116 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c671a718-7b3f-49ae-b29a-b6e14b5cbeb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:57.060772 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c671a718-7b3f-49ae-b29a-b6e14b5cbeb0 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:57.061087 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c671a718-7b3f-49ae-b29a-b6e14b5cbeb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:57.061357 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c671a718-7b3f-49ae-b29a-b6e14b5cbeb0 None admin] PATH_INFO: `/v3/projects/197e55aa210c42abbf12e585a968f8c5` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:57.063243 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c671a718-7b3f-49ae-b29a-b6e14b5cbeb0 None admin] RBAC: Authorizing `identity:get_project(project_id=197e55aa210c42abbf12e585a968f8c5)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:57.065013 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c671a718-7b3f-49ae-b29a-b6e14b5cbeb0 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:57.067109 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 192/403] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 6 19:58:56 2022] GET /identity/v3/projects/197e55aa210c42abbf12e585a968f8c5 => generated 298 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:57.072764 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c671a718-7b3f-49ae-b29a-b6e14b5cbeb0 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:57.113664 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f328e7-004c-48f3-80bd-c0e4fa8834e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:57.114787 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f328e7-004c-48f3-80bd-c0e4fa8834e7 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:57.117390 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f328e7-004c-48f3-80bd-c0e4fa8834e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:57.117390 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f328e7-004c-48f3-80bd-c0e4fa8834e7 None admin] PATH_INFO: `/v3/projects/197e55aa210c42abbf12e585a968f8c5` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:57.118266 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f328e7-004c-48f3-80bd-c0e4fa8834e7 None admin] RBAC: Authorizing `identity:get_project(project_id=197e55aa210c42abbf12e585a968f8c5)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:57.121141 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f328e7-004c-48f3-80bd-c0e4fa8834e7 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:57.124575 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 212/404] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 6 19:58:56 2022] GET /identity/v3/projects/197e55aa210c42abbf12e585a968f8c5 => generated 298 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:57.124575 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788ff638-feda-4807-996a-cdccc88b18cf None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:57.124575 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788ff638-feda-4807-996a-cdccc88b18cf None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:57.125093 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788ff638-feda-4807-996a-cdccc88b18cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:57.157576 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-788ff638-feda-4807-996a-cdccc88b18cf None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 19:58:57.159303 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-788ff638-feda-4807-996a-cdccc88b18cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 19:58:57.257555 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 213/405] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 19:58:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 19:58:57.258674 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788ff638-feda-4807-996a-cdccc88b18cf None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:57.286816 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c671a718-7b3f-49ae-b29a-b6e14b5cbeb0 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:57.316079 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:57.392838 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85bf1bf-bec5-4eb5-95ad-a236f13f2893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:57.394367 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85bf1bf-bec5-4eb5-95ad-a236f13f2893 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:57.395504 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85bf1bf-bec5-4eb5-95ad-a236f13f2893 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:57.396248 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85bf1bf-bec5-4eb5-95ad-a236f13f2893 None admin] PATH_INFO: `/v3/groups/6a12d79ef6944c129aeb6eccf1643416` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:57.399847 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85bf1bf-bec5-4eb5-95ad-a236f13f2893 None admin] RBAC: Authorizing `identity:get_group(group_id=6a12d79ef6944c129aeb6eccf1643416)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:57.402182 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788ff638-feda-4807-996a-cdccc88b18cf None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:57.403171 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85bf1bf-bec5-4eb5-95ad-a236f13f2893 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:57.408927 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 193/406] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Sep 6 19:58:57 2022] GET /identity/v3/groups/6a12d79ef6944c129aeb6eccf1643416 => generated 226 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:57.410954 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85bf1bf-bec5-4eb5-95ad-a236f13f2893 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:57.474890 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffde2835-8d98-44a3-9d9d-68fb8ef1daef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:57.477024 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffde2835-8d98-44a3-9d9d-68fb8ef1daef None admin] REQUEST_METHOD: `PUT` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:57.479597 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffde2835-8d98-44a3-9d9d-68fb8ef1daef None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:57.480438 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffde2835-8d98-44a3-9d9d-68fb8ef1daef None admin] PATH_INFO: `/v3/projects/197e55aa210c42abbf12e585a968f8c5/groups/6a12d79ef6944c129aeb6eccf1643416/roles/b02170460f054c259e825ab6e35f8537` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:57.485195 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffde2835-8d98-44a3-9d9d-68fb8ef1daef None admin] RBAC: Authorizing `identity:create_grant(project_id=197e55aa210c42abbf12e585a968f8c5, group_id=6a12d79ef6944c129aeb6eccf1643416, role_id=b02170460f054c259e825ab6e35f8537)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:57.488401 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffde2835-8d98-44a3-9d9d-68fb8ef1daef None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:57.504488 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:57.517922 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 214/407] 10.222.0.20 () {58 vars in 1442 bytes} [Tue Sep 6 19:58:57 2022] PUT /identity/v3/projects/197e55aa210c42abbf12e585a968f8c5/groups/6a12d79ef6944c129aeb6eccf1643416/roles/b02170460f054c259e825ab6e35f8537 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 19:58:57.519376 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffde2835-8d98-44a3-9d9d-68fb8ef1daef None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:57.653147 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85bf1bf-bec5-4eb5-95ad-a236f13f2893 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:57.728121 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907350e9-3944-4aa5-82ee-a73f595bf02e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:57.729772 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907350e9-3944-4aa5-82ee-a73f595bf02e None admin] REQUEST_METHOD: `PUT` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:57.730535 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907350e9-3944-4aa5-82ee-a73f595bf02e None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:57.731404 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907350e9-3944-4aa5-82ee-a73f595bf02e None admin] PATH_INFO: `/v3/projects/197e55aa210c42abbf12e585a968f8c5/groups/6a12d79ef6944c129aeb6eccf1643416/roles/4eea39e24b06431aa357053fe7907a3d` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:57.736040 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907350e9-3944-4aa5-82ee-a73f595bf02e None admin] RBAC: Authorizing `identity:create_grant(project_id=197e55aa210c42abbf12e585a968f8c5, group_id=6a12d79ef6944c129aeb6eccf1643416, role_id=4eea39e24b06431aa357053fe7907a3d)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:57.738699 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907350e9-3944-4aa5-82ee-a73f595bf02e None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:57.739296 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:57.744031 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffde2835-8d98-44a3-9d9d-68fb8ef1daef None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:57.773711 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 194/408] 10.222.0.20 () {58 vars in 1442 bytes} [Tue Sep 6 19:58:57 2022] PUT /identity/v3/projects/197e55aa210c42abbf12e585a968f8c5/groups/6a12d79ef6944c129aeb6eccf1643416/roles/4eea39e24b06431aa357053fe7907a3d => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 19:58:57.774789 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907350e9-3944-4aa5-82ee-a73f595bf02e None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:57.794407 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297f0f12-1f5e-4c29-8114-ef9dbd3c9dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:57.794407 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297f0f12-1f5e-4c29-8114-ef9dbd3c9dbb None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:57.794407 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297f0f12-1f5e-4c29-8114-ef9dbd3c9dbb None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:57.794407 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297f0f12-1f5e-4c29-8114-ef9dbd3c9dbb None admin] PATH_INFO: `/v3/projects/22a00d6dfd004be4aa2f9727f02e69ab` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:57.795552 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297f0f12-1f5e-4c29-8114-ef9dbd3c9dbb None admin] RBAC: Authorizing `identity:get_project(project_id=22a00d6dfd004be4aa2f9727f02e69ab)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:57.798310 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297f0f12-1f5e-4c29-8114-ef9dbd3c9dbb None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:57.799696 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 215/409] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 6 19:58:57 2022] GET /identity/v3/projects/22a00d6dfd004be4aa2f9727f02e69ab => generated 302 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:57.809646 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297f0f12-1f5e-4c29-8114-ef9dbd3c9dbb None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:57.942999 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907350e9-3944-4aa5-82ee-a73f595bf02e None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:57.950382 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297f0f12-1f5e-4c29-8114-ef9dbd3c9dbb None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:57.951790 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:57.992388 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52417e09-5c68-469b-aea9-a121b9d84991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:57.993232 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52417e09-5c68-469b-aea9-a121b9d84991 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:57.993539 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52417e09-5c68-469b-aea9-a121b9d84991 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:57.993864 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52417e09-5c68-469b-aea9-a121b9d84991 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:57.994886 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dfd991-ddab-4b40-9dc0-4957ac94701c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:57.995671 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dfd991-ddab-4b40-9dc0-4957ac94701c None admin] REQUEST_METHOD: `PUT` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:57.996054 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dfd991-ddab-4b40-9dc0-4957ac94701c None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:57.996335 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dfd991-ddab-4b40-9dc0-4957ac94701c None admin] PATH_INFO: `/v3/projects/22a00d6dfd004be4aa2f9727f02e69ab/groups/6a12d79ef6944c129aeb6eccf1643416/roles/b02170460f054c259e825ab6e35f8537` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:57.998929 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dfd991-ddab-4b40-9dc0-4957ac94701c None admin] RBAC: Authorizing `identity:create_grant(project_id=22a00d6dfd004be4aa2f9727f02e69ab, group_id=6a12d79ef6944c129aeb6eccf1643416, role_id=b02170460f054c259e825ab6e35f8537)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:58.000769 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dfd991-ddab-4b40-9dc0-4957ac94701c None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:58.009125 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52417e09-5c68-469b-aea9-a121b9d84991 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 19:58:58.017295 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-52417e09-5c68-469b-aea9-a121b9d84991 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 19:58:58.017295 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:58:58.017295 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:58:58.017295 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:58:58.017295 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:58:58.017295 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:58:58.017295 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:58:58.017295 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:58:58.017295 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:58:58.017295 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:58:58.017295 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:58:58.017295 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:58:58.017295 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 19:58:58.017295 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 19:58:58.017295 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 19:58:58.017295 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 19:58:58.019280 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 19:58:58.019280 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 19:58:58.019280 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 19:58:58.019280 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 19:58:58.019280 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:58:58.019280 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:58:58.019280 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:58:58.019280 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:58:58.019280 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:58:58.019280 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:58:58.019280 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:58:58.019280 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 19:58:58.019280 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:58:58.019280 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:58:58.019280 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:58:58.019280 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:58:58.020243 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:58:58.020243 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:58:58.020243 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:58:58.020243 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:58:58.020243 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 19:58:58.020243 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 19:58:58.020243 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 19:58:58.020243 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 19:58:58.020243 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 19:58:58.020243 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 19:58:58.020243 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 19:58:58.020243 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 19:58:58.021006 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 216/410] 10.222.0.20 () {58 vars in 1442 bytes} [Tue Sep 6 19:58:57 2022] PUT /identity/v3/projects/22a00d6dfd004be4aa2f9727f02e69ab/groups/6a12d79ef6944c129aeb6eccf1643416/roles/b02170460f054c259e825ab6e35f8537 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 19:58:58.021227 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 195/411] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Sep 6 19:58:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:58:58.029213 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dfd991-ddab-4b40-9dc0-4957ac94701c None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:58.085912 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:58.253381 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dfd991-ddab-4b40-9dc0-4957ac94701c None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:58.312104 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a0236c-1544-4132-81fe-3c42e28219dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:58.313481 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a0236c-1544-4132-81fe-3c42e28219dc None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:58.313893 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a0236c-1544-4132-81fe-3c42e28219dc None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:58.314267 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a0236c-1544-4132-81fe-3c42e28219dc None admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:58.315109 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a0236c-1544-4132-81fe-3c42e28219dc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:58:58.315526 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a0236c-1544-4132-81fe-3c42e28219dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:58.317106 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a0236c-1544-4132-81fe-3c42e28219dc None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:58.328450 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 217/412] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 6 19:58:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:58.334486 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52417e09-5c68-469b-aea9-a121b9d84991 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:58.497224 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73240f5a-ff0f-4c1d-81e3-1b1dc83ac0ca None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:58.497558 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73240f5a-ff0f-4c1d-81e3-1b1dc83ac0ca None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:58.497827 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73240f5a-ff0f-4c1d-81e3-1b1dc83ac0ca None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:58.499552 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 218/413] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 19:58:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 19:58:58.506478 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e50f1c5-6c2f-466f-8d3b-a6bb97e0e774 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:58.506777 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e50f1c5-6c2f-466f-8d3b-a6bb97e0e774 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:58.507033 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e50f1c5-6c2f-466f-8d3b-a6bb97e0e774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:58.511500 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52417e09-5c68-469b-aea9-a121b9d84991 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:58.537114 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-9e50f1c5-6c2f-466f-8d3b-a6bb97e0e774 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 19:58:58.538249 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-9e50f1c5-6c2f-466f-8d3b-a6bb97e0e774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 19:58:58.586484 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362dada2-1622-4849-9790-80fe7b88cef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:58.587197 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362dada2-1622-4849-9790-80fe7b88cef6 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:58.587480 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362dada2-1622-4849-9790-80fe7b88cef6 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:58.587819 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362dada2-1622-4849-9790-80fe7b88cef6 None admin] PATH_INFO: `/v3/groups/c2db17bd65f74cfb8e12147856fca032` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:58.589165 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362dada2-1622-4849-9790-80fe7b88cef6 None admin] RBAC: Authorizing `identity:get_group(group_id=c2db17bd65f74cfb8e12147856fca032)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:58.590767 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362dada2-1622-4849-9790-80fe7b88cef6 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:58.592773 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 196/414] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Sep 6 19:58:58 2022] GET /identity/v3/groups/c2db17bd65f74cfb8e12147856fca032 => generated 229 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:58.600202 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362dada2-1622-4849-9790-80fe7b88cef6 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:58.619150 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 219/415] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 19:58:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 19:58:58.646615 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:58.765141 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362dada2-1622-4849-9790-80fe7b88cef6 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:58.826168 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723c0a14-d62d-4a84-a9dc-0c8aceb809ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:58.827826 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723c0a14-d62d-4a84-a9dc-0c8aceb809ce None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:58.829183 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723c0a14-d62d-4a84-a9dc-0c8aceb809ce None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:58.829803 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723c0a14-d62d-4a84-a9dc-0c8aceb809ce None admin] PATH_INFO: `/v3/projects/a630b7748ecd4893b8db1b6c1dc4efe9` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:58.838221 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723c0a14-d62d-4a84-a9dc-0c8aceb809ce None admin] RBAC: Authorizing `identity:get_project(project_id=a630b7748ecd4893b8db1b6c1dc4efe9)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:58.840430 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723c0a14-d62d-4a84-a9dc-0c8aceb809ce None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:58.843317 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 197/416] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 6 19:58:58 2022] GET /identity/v3/projects/a630b7748ecd4893b8db1b6c1dc4efe9 => generated 344 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:58.849646 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e50f1c5-6c2f-466f-8d3b-a6bb97e0e774 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:58.896790 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:59.016014 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e50f1c5-6c2f-466f-8d3b-a6bb97e0e774 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:59.056927 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afad87a-0c81-44e4-ae1a-9d94e16611a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:59.057633 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afad87a-0c81-44e4-ae1a-9d94e16611a0 None admin] REQUEST_METHOD: `PUT` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:59.057929 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afad87a-0c81-44e4-ae1a-9d94e16611a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:59.058241 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afad87a-0c81-44e4-ae1a-9d94e16611a0 None admin] PATH_INFO: `/v3/projects/a630b7748ecd4893b8db1b6c1dc4efe9/groups/c2db17bd65f74cfb8e12147856fca032/roles/e0ab41d8ae22474688b5361bcc75e0f0` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:59.060437 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afad87a-0c81-44e4-ae1a-9d94e16611a0 None admin] RBAC: Authorizing `identity:create_grant(project_id=a630b7748ecd4893b8db1b6c1dc4efe9, group_id=c2db17bd65f74cfb8e12147856fca032, role_id=e0ab41d8ae22474688b5361bcc75e0f0)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:59.061977 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afad87a-0c81-44e4-ae1a-9d94e16611a0 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:59.084073 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 220/417] 10.222.0.20 () {58 vars in 1442 bytes} [Tue Sep 6 19:58:58 2022] PUT /identity/v3/projects/a630b7748ecd4893b8db1b6c1dc4efe9/groups/c2db17bd65f74cfb8e12147856fca032/roles/e0ab41d8ae22474688b5361bcc75e0f0 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 19:58:59.206802 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723c0a14-d62d-4a84-a9dc-0c8aceb809ce None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:59.366414 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723c0a14-d62d-4a84-a9dc-0c8aceb809ce None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:59.368279 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:59.410953 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd0c085-ff0d-4f5e-bd0e-974e84f61f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:59.411606 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd0c085-ff0d-4f5e-bd0e-974e84f61f44 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:59.411886 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd0c085-ff0d-4f5e-bd0e-974e84f61f44 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:59.412158 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd0c085-ff0d-4f5e-bd0e-974e84f61f44 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:59.426008 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd0c085-ff0d-4f5e-bd0e-974e84f61f44 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 19:58:59.432041 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-cdd0c085-ff0d-4f5e-bd0e-974e84f61f44 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 19:58:59.432041 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:58:59.432041 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:58:59.432041 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:58:59.432041 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:58:59.432041 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:58:59.432041 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:58:59.432041 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:58:59.432041 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:58:59.432041 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:58:59.432041 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:58:59.432041 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:58:59.432041 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 19:58:59.432041 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 19:58:59.432041 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 19:58:59.432041 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 19:58:59.433008 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 19:58:59.433008 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 19:58:59.433008 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 19:58:59.433008 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 19:58:59.433008 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:58:59.433008 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:58:59.433008 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:58:59.433008 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:58:59.433008 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:58:59.433008 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:58:59.433008 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:58:59.433008 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 19:58:59.433008 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:58:59.433008 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:58:59.433008 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:58:59.433008 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:58:59.434561 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:58:59.434561 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:58:59.434561 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:58:59.434561 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:58:59.434561 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 19:58:59.434561 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 19:58:59.434561 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 19:58:59.434561 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 19:58:59.434561 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 19:58:59.434561 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 19:58:59.434561 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 19:58:59.434561 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 19:58:59.434561 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 198/418] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 6 19:58:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:58:59.439802 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afad87a-0c81-44e4-ae1a-9d94e16611a0 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:59.541084 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:58:59.572973 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afad87a-0c81-44e4-ae1a-9d94e16611a0 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:58:59.626076 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b0fa95-38ee-4347-b067-f3a19833ee8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:58:59.626628 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b0fa95-38ee-4347-b067-f3a19833ee8a None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:58:59.626994 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b0fa95-38ee-4347-b067-f3a19833ee8a None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:58:59.627271 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b0fa95-38ee-4347-b067-f3a19833ee8a None admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:58:59.627983 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b0fa95-38ee-4347-b067-f3a19833ee8a None admin] RBAC: Adding query filter params (name=member) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:58:59.628289 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b0fa95-38ee-4347-b067-f3a19833ee8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:58:59.630144 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b0fa95-38ee-4347-b067-f3a19833ee8a None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:58:59.643396 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 221/419] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Sep 6 19:58:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:58:59.651590 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd0c085-ff0d-4f5e-bd0e-974e84f61f44 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:58:59.973735 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd0c085-ff0d-4f5e-bd0e-974e84f61f44 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:00.084640 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd8d908-6ebf-459a-8244-485ed27b5ddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:00.085753 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd8d908-6ebf-459a-8244-485ed27b5ddb None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:00.086353 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd8d908-6ebf-459a-8244-485ed27b5ddb None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:00.086884 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd8d908-6ebf-459a-8244-485ed27b5ddb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:00.190613 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:00.302937 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd8d908-6ebf-459a-8244-485ed27b5ddb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:00.305224 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd8d908-6ebf-459a-8244-485ed27b5ddb None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:00.583566 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 199/420] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 6 19:58:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 933 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 19:59:00.596047 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b0fa95-38ee-4347-b067-f3a19833ee8a None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:00.828020 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbd2d57-5a27-4663-92ff-041f735070bb None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:00.828800 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbd2d57-5a27-4663-92ff-041f735070bb None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:00.829288 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbd2d57-5a27-4663-92ff-041f735070bb None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:00.831207 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 200/421] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 19:59:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 19:59:00.838326 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6208ec45-80d9-41bf-872b-c1a7352afbcf None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:00.839216 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6208ec45-80d9-41bf-872b-c1a7352afbcf None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:00.840000 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6208ec45-80d9-41bf-872b-c1a7352afbcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:00.886274 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b0fa95-38ee-4347-b067-f3a19833ee8a None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:00.907747 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-6208ec45-80d9-41bf-872b-c1a7352afbcf None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 19:59:00.910152 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-6208ec45-80d9-41bf-872b-c1a7352afbcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 19:59:00.972906 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2539cf0c-03f7-4102-a9e8-31ad6d4b7e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:00.974080 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2539cf0c-03f7-4102-a9e8-31ad6d4b7e0a None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:00.974569 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2539cf0c-03f7-4102-a9e8-31ad6d4b7e0a None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:00.975050 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2539cf0c-03f7-4102-a9e8-31ad6d4b7e0a None admin] PATH_INFO: `/v3/projects/22a00d6dfd004be4aa2f9727f02e69ab` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:00.980834 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2539cf0c-03f7-4102-a9e8-31ad6d4b7e0a None admin] RBAC: Authorizing `identity:get_project(project_id=22a00d6dfd004be4aa2f9727f02e69ab)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:00.985823 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2539cf0c-03f7-4102-a9e8-31ad6d4b7e0a None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:00.989107 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 222/422] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 6 19:59:00 2022] GET /identity/v3/projects/22a00d6dfd004be4aa2f9727f02e69ab => generated 302 bytes in 395 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:00.998062 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2539cf0c-03f7-4102-a9e8-31ad6d4b7e0a None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:01.042036 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:01.122229 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:01.152992 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 201/423] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 19:59:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 19:59:01.256265 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2539cf0c-03f7-4102-a9e8-31ad6d4b7e0a None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:01.328359 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b56a91c-0330-4e30-8969-77dc39dd3a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:01.329510 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b56a91c-0330-4e30-8969-77dc39dd3a78 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:01.330061 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b56a91c-0330-4e30-8969-77dc39dd3a78 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:01.330552 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b56a91c-0330-4e30-8969-77dc39dd3a78 None admin] PATH_INFO: `/v3/groups/6a12d79ef6944c129aeb6eccf1643416` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:01.332785 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b56a91c-0330-4e30-8969-77dc39dd3a78 None admin] RBAC: Authorizing `identity:get_group(group_id=6a12d79ef6944c129aeb6eccf1643416)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:01.335689 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b56a91c-0330-4e30-8969-77dc39dd3a78 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:01.339377 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 223/424] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Sep 6 19:59:00 2022] GET /identity/v3/groups/6a12d79ef6944c129aeb6eccf1643416 => generated 226 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:01.347080 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6208ec45-80d9-41bf-872b-c1a7352afbcf None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:01.483807 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bfc4bd-249f-45d4-a5bb-8364dcf2ea67 None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:01.484123 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bfc4bd-249f-45d4-a5bb-8364dcf2ea67 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:01.484348 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bfc4bd-249f-45d4-a5bb-8364dcf2ea67 None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:01.486686 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 224/425] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 19:59:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 19:59:01.492794 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570dba43-af01-45cc-bf87-0bf2668a6554 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:01.493105 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570dba43-af01-45cc-bf87-0bf2668a6554 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:01.493349 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570dba43-af01-45cc-bf87-0bf2668a6554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:01.538945 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-570dba43-af01-45cc-bf87-0bf2668a6554 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 19:59:01.543515 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-570dba43-af01-45cc-bf87-0bf2668a6554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 19:59:01.589459 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6208ec45-80d9-41bf-872b-c1a7352afbcf None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:01.679176 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01f09ed-2525-4953-a93f-c861f0f3586e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:01.680419 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01f09ed-2525-4953-a93f-c861f0f3586e None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:01.680907 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01f09ed-2525-4953-a93f-c861f0f3586e None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:01.681389 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01f09ed-2525-4953-a93f-c861f0f3586e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:01.683400 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01f09ed-2525-4953-a93f-c861f0f3586e None admin] RBAC: Adding query filter params (group.id=6a12d79ef6944c129aeb6eccf1643416, role.id=4eea39e24b06431aa357053fe7907a3d, scope.project.id=22a00d6dfd004be4aa2f9727f02e69ab) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:01.683957 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01f09ed-2525-4953-a93f-c861f0f3586e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:01.687674 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01f09ed-2525-4953-a93f-c861f0f3586e None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:01.704532 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 202/426] 10.222.0.20 () {56 vars in 1376 bytes} [Tue Sep 6 19:59:01 2022] GET /identity/v3/role_assignments?group.id=6a12d79ef6944c129aeb6eccf1643416&scope.project.id=22a00d6dfd004be4aa2f9727f02e69ab&role.id=4eea39e24b06431aa357053fe7907a3d => generated 611 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 19:59:01.706333 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01f09ed-2525-4953-a93f-c861f0f3586e None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:01.728319 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 225/427] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 19:59:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 19:59:01.822744 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8034a26-d493-4fed-b4d8-b34030dfd34f None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:01.823159 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8034a26-d493-4fed-b4d8-b34030dfd34f None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:01.823657 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8034a26-d493-4fed-b4d8-b34030dfd34f None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:01.827096 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 226/428] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 19:59:01 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 19:59:01.834745 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce2750a-0559-4966-8c68-97145918b83b None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:01.836017 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce2750a-0559-4966-8c68-97145918b83b None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:01.836550 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce2750a-0559-4966-8c68-97145918b83b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:01.876954 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:01.878708 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:01.892650 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-4ce2750a-0559-4966-8c68-97145918b83b None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 19:59:01.894389 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-4ce2750a-0559-4966-8c68-97145918b83b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 19:59:01.993707 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01f09ed-2525-4953-a93f-c861f0f3586e None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:02.048678 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 227/429] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 19:59:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 19:59:02.053667 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9ef229-e38e-4151-9a59-bd44fb946734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:02.053667 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9ef229-e38e-4151-9a59-bd44fb946734 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:02.053667 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9ef229-e38e-4151-9a59-bd44fb946734 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:02.053667 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9ef229-e38e-4151-9a59-bd44fb946734 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:02.068137 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9ef229-e38e-4151-9a59-bd44fb946734 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 19:59:02.074020 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-7e9ef229-e38e-4151-9a59-bd44fb946734 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 19:59:02.074020 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:02.074020 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:02.074020 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:02.074020 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:02.074020 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:02.074020 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:02.074020 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:02.074020 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:02.074020 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:02.074020 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:02.074020 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:02.074020 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 19:59:02.074020 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 19:59:02.074020 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 19:59:02.074020 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 19:59:02.075045 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 19:59:02.075045 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 19:59:02.075045 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 19:59:02.075045 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 19:59:02.075045 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:02.075045 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:02.075045 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:02.075045 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:02.075045 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:02.075045 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:02.075045 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:02.075045 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:02.075045 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:02.075045 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:02.075045 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:02.075045 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:02.075872 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:02.075872 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:02.075872 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:02.075872 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:02.075872 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 19:59:02.075872 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 19:59:02.075872 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 19:59:02.075872 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 19:59:02.075872 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 19:59:02.075872 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 19:59:02.075872 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 19:59:02.075872 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 19:59:02.078825 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 203/430] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Sep 6 19:59:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 374 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:59:02.085148 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce2750a-0559-4966-8c68-97145918b83b None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:02.236173 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce2750a-0559-4966-8c68-97145918b83b None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:02.291344 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c3b2d2-37fd-4222-891f-529121706deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:02.291981 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c3b2d2-37fd-4222-891f-529121706deb None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:02.292421 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c3b2d2-37fd-4222-891f-529121706deb None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:02.292720 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c3b2d2-37fd-4222-891f-529121706deb None admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:02.293501 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c3b2d2-37fd-4222-891f-529121706deb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:02.293895 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c3b2d2-37fd-4222-891f-529121706deb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:02.295470 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c3b2d2-37fd-4222-891f-529121706deb None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:02.299872 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:02.308499 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 228/431] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 6 19:59:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:02.315554 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9ef229-e38e-4151-9a59-bd44fb946734 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:02.337985 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c3b2d2-37fd-4222-891f-529121706deb None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:02.495763 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c3b2d2-37fd-4222-891f-529121706deb None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:02.523134 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9ef229-e38e-4151-9a59-bd44fb946734 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:02.559775 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:02.561305 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81eb91e-d6cd-4b2a-9657-11b704e37b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:02.561846 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81eb91e-d6cd-4b2a-9657-11b704e37b59 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:02.562030 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81eb91e-d6cd-4b2a-9657-11b704e37b59 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:02.562440 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81eb91e-d6cd-4b2a-9657-11b704e37b59 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:02.583697 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81eb91e-d6cd-4b2a-9657-11b704e37b59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 19:59:02.589828 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e655d889-f122-400e-b99b-54b6fca5ec61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:02.590363 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e655d889-f122-400e-b99b-54b6fca5ec61 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:02.590640 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e655d889-f122-400e-b99b-54b6fca5ec61 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:02.590926 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e655d889-f122-400e-b99b-54b6fca5ec61 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:02.594732 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-b81eb91e-d6cd-4b2a-9657-11b704e37b59 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 19:59:02.594732 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:02.594732 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:02.594732 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:02.594732 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:02.594732 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:02.594732 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:02.594732 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:02.594732 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:02.594732 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:02.594732 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:02.594732 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:02.594732 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 19:59:02.594732 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 19:59:02.594732 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 19:59:02.594732 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 19:59:02.598856 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 19:59:02.598856 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 19:59:02.598856 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 19:59:02.598856 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 19:59:02.598856 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:02.598856 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:02.598856 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:02.598856 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:02.598856 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:02.598856 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:02.598856 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:02.598856 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:02.598856 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:02.598856 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:02.598856 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:02.598856 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:02.600724 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:02.600724 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:02.600724 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:02.600724 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:02.600724 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 19:59:02.600724 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 19:59:02.600724 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 19:59:02.600724 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 19:59:02.600724 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 19:59:02.600724 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 19:59:02.600724 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 19:59:02.600724 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 19:59:02.600724 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 229/432] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Sep 6 19:59:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 19:59:02.600724 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81eb91e-d6cd-4b2a-9657-11b704e37b59 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:02.706699 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:02.727726 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e655d889-f122-400e-b99b-54b6fca5ec61 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:02.730267 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e655d889-f122-400e-b99b-54b6fca5ec61 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:02.789340 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81eb91e-d6cd-4b2a-9657-11b704e37b59 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:02.855407 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502973e1-2fc8-4f66-b085-91df4eb51c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:02.856569 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502973e1-2fc8-4f66-b085-91df4eb51c29 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:02.857224 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502973e1-2fc8-4f66-b085-91df4eb51c29 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:02.857910 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502973e1-2fc8-4f66-b085-91df4eb51c29 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:02.876738 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502973e1-2fc8-4f66-b085-91df4eb51c29 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 19:59:02.886477 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-502973e1-2fc8-4f66-b085-91df4eb51c29 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 19:59:02.886477 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:02.886477 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:02.886477 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:02.886477 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:02.886477 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:02.886477 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:02.886477 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:02.886477 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:02.886477 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:02.886477 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:02.886477 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:02.886477 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 19:59:02.886477 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 19:59:02.886477 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 19:59:02.886477 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 19:59:02.888045 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 19:59:02.888045 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 19:59:02.888045 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 19:59:02.888045 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 19:59:02.888045 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:02.888045 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:02.888045 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:02.888045 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:02.888045 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:02.888045 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:02.888045 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:02.888045 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:02.888045 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:02.888045 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:02.888045 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:02.888045 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:02.888900 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:02.888900 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:02.888900 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:02.888900 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:02.888900 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 19:59:02.888900 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 19:59:02.888900 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 19:59:02.888900 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 19:59:02.888900 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 19:59:02.888900 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 19:59:02.888900 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 19:59:02.888900 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 19:59:02.888900 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 230/433] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 6 19:59:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:59:02.889784 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502973e1-2fc8-4f66-b085-91df4eb51c29 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:02.920624 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 204/434] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 6 19:59:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 606 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 19:59:02.923095 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c073442-0d4d-4509-bf63-3a9826504c92 None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:02.923570 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c073442-0d4d-4509-bf63-3a9826504c92 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:02.924050 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c073442-0d4d-4509-bf63-3a9826504c92 None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:02.928055 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 205/435] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 19:59:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 19:59:02.928714 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c073442-0d4d-4509-bf63-3a9826504c92 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:02.952724 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:03.030094 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502973e1-2fc8-4f66-b085-91df4eb51c29 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:03.052553 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:03.079196 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00282e34-78fb-4115-b91b-135177f02dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:03.080250 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00282e34-78fb-4115-b91b-135177f02dec None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:03.080653 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00282e34-78fb-4115-b91b-135177f02dec None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:03.080969 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00282e34-78fb-4115-b91b-135177f02dec None admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:03.082046 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00282e34-78fb-4115-b91b-135177f02dec None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:03.082530 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00282e34-78fb-4115-b91b-135177f02dec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:03.084891 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00282e34-78fb-4115-b91b-135177f02dec None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:03.098668 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 231/436] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 6 19:59:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:03.099980 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00282e34-78fb-4115-b91b-135177f02dec None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:03.113386 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c073442-0d4d-4509-bf63-3a9826504c92 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:03.164015 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e6c2f1-e93e-4006-9089-82345a13cee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:03.165234 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e6c2f1-e93e-4006-9089-82345a13cee2 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:03.165766 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e6c2f1-e93e-4006-9089-82345a13cee2 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:03.166236 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e6c2f1-e93e-4006-9089-82345a13cee2 None admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:03.167894 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e6c2f1-e93e-4006-9089-82345a13cee2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:03.168518 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e6c2f1-e93e-4006-9089-82345a13cee2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:03.171518 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e6c2f1-e93e-4006-9089-82345a13cee2 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:03.188165 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 206/437] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Sep 6 19:59:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:03.190551 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2b96eb-8ba1-4291-a3c2-a8d4bd94cdd3 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:03.191084 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2b96eb-8ba1-4291-a3c2-a8d4bd94cdd3 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:03.191645 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2b96eb-8ba1-4291-a3c2-a8d4bd94cdd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:03.221873 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-0a2b96eb-8ba1-4291-a3c2-a8d4bd94cdd3 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 19:59:03.223293 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-0a2b96eb-8ba1-4291-a3c2-a8d4bd94cdd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 19:59:03.227727 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00282e34-78fb-4115-b91b-135177f02dec None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:03.272973 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecb0109-1321-47b9-af64-669f2f0be513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:03.273521 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecb0109-1321-47b9-af64-669f2f0be513 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:03.273758 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecb0109-1321-47b9-af64-669f2f0be513 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:03.274224 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecb0109-1321-47b9-af64-669f2f0be513 None admin] PATH_INFO: `/v3/projects/197e55aa210c42abbf12e585a968f8c5` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:03.275984 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecb0109-1321-47b9-af64-669f2f0be513 None admin] RBAC: Authorizing `identity:get_project(project_id=197e55aa210c42abbf12e585a968f8c5)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:03.277604 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecb0109-1321-47b9-af64-669f2f0be513 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:03.279779 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 232/438] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 6 19:59:03 2022] GET /identity/v3/projects/197e55aa210c42abbf12e585a968f8c5 => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:03.281131 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecb0109-1321-47b9-af64-669f2f0be513 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:03.353503 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 207/439] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 19:59:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 19:59:03.354719 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2b96eb-8ba1-4291-a3c2-a8d4bd94cdd3 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:03.374066 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:03.378233 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:03.485700 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecb0109-1321-47b9-af64-669f2f0be513 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:03.561393 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b753897-dbed-407b-b21b-81321c6b12fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:03.562466 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b753897-dbed-407b-b21b-81321c6b12fa None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:03.563482 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b753897-dbed-407b-b21b-81321c6b12fa None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:03.564022 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b753897-dbed-407b-b21b-81321c6b12fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:03.571436 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2b96eb-8ba1-4291-a3c2-a8d4bd94cdd3 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:03.631217 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d907cb-8b91-4cc6-8f90-3df70664f872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:03.632161 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d907cb-8b91-4cc6-8f90-3df70664f872 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:03.632614 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d907cb-8b91-4cc6-8f90-3df70664f872 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:03.633219 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d907cb-8b91-4cc6-8f90-3df70664f872 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:03.707478 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b753897-dbed-407b-b21b-81321c6b12fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:03.712044 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b753897-dbed-407b-b21b-81321c6b12fa None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:03.773646 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d907cb-8b91-4cc6-8f90-3df70664f872 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:03.775488 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d907cb-8b91-4cc6-8f90-3df70664f872 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:03.791658 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:03.817927 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:03.899926 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 233/440] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 6 19:59:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 619 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 19:59:03.901236 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b753897-dbed-407b-b21b-81321c6b12fa None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:03.940762 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 208/441] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 6 19:59:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 587 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 19:59:03.942403 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d907cb-8b91-4cc6-8f90-3df70664f872 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:04.057655 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b753897-dbed-407b-b21b-81321c6b12fa None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:04.086850 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d907cb-8b91-4cc6-8f90-3df70664f872 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:04.100713 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:04.136336 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59c650f-8622-464b-a063-d6c031bce72b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:04.137793 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59c650f-8622-464b-a063-d6c031bce72b None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:04.138236 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59c650f-8622-464b-a063-d6c031bce72b None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:04.138737 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59c650f-8622-464b-a063-d6c031bce72b None admin] PATH_INFO: `/v3/groups/6a12d79ef6944c129aeb6eccf1643416` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:04.141645 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59c650f-8622-464b-a063-d6c031bce72b None admin] RBAC: Authorizing `identity:get_group(group_id=6a12d79ef6944c129aeb6eccf1643416)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:04.144504 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59c650f-8622-464b-a063-d6c031bce72b None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:04.154192 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 234/442] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Sep 6 19:59:03 2022] GET /identity/v3/groups/6a12d79ef6944c129aeb6eccf1643416 => generated 226 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:04.154192 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59c650f-8622-464b-a063-d6c031bce72b None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:04.154913 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f91874-90ed-4485-8de1-15bdb8247f54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:04.156015 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f91874-90ed-4485-8de1-15bdb8247f54 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:04.157978 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f91874-90ed-4485-8de1-15bdb8247f54 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:04.158474 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f91874-90ed-4485-8de1-15bdb8247f54 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:04.175955 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:04.182406 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f91874-90ed-4485-8de1-15bdb8247f54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 19:59:04.192652 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-c7f91874-90ed-4485-8de1-15bdb8247f54 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 19:59:04.192652 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:04.192652 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:04.192652 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:04.192652 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:04.192652 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:04.192652 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:04.192652 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:04.192652 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:04.192652 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:04.192652 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:04.192652 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:04.192652 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 19:59:04.192652 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 19:59:04.192652 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 19:59:04.192652 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 19:59:04.193993 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 19:59:04.193993 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 19:59:04.193993 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 19:59:04.193993 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 19:59:04.193993 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:04.193993 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:04.193993 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:04.193993 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:04.193993 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:04.193993 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:04.193993 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:04.193993 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:04.193993 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:04.193993 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:04.193993 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:04.193993 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:04.194834 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:04.194834 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:04.194834 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:04.194834 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:04.194834 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 19:59:04.194834 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 19:59:04.194834 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 19:59:04.194834 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 19:59:04.194834 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 19:59:04.194834 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 19:59:04.194834 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 19:59:04.194834 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 19:59:04.197813 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 209/443] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Sep 6 19:59:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:59:04.199191 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f91874-90ed-4485-8de1-15bdb8247f54 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:04.283785 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59c650f-8622-464b-a063-d6c031bce72b None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:04.338652 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f96ac6-bb3f-4e81-abca-0cbde10a5668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:04.339428 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f96ac6-bb3f-4e81-abca-0cbde10a5668 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:04.339775 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f96ac6-bb3f-4e81-abca-0cbde10a5668 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:04.340119 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f96ac6-bb3f-4e81-abca-0cbde10a5668 None admin] PATH_INFO: `/v3/projects/22a00d6dfd004be4aa2f9727f02e69ab` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:04.341542 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f96ac6-bb3f-4e81-abca-0cbde10a5668 None admin] RBAC: Authorizing `identity:get_project(project_id=22a00d6dfd004be4aa2f9727f02e69ab)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:04.343535 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f96ac6-bb3f-4e81-abca-0cbde10a5668 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:04.346439 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 235/444] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 6 19:59:04 2022] GET /identity/v3/projects/22a00d6dfd004be4aa2f9727f02e69ab => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:04.348109 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f96ac6-bb3f-4e81-abca-0cbde10a5668 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:04.349683 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f91874-90ed-4485-8de1-15bdb8247f54 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:04.405299 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a9b788-c927-427d-9e2f-61e86095ae6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:04.405951 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a9b788-c927-427d-9e2f-61e86095ae6d None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:04.406239 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a9b788-c927-427d-9e2f-61e86095ae6d None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:04.407211 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a9b788-c927-427d-9e2f-61e86095ae6d None admin] PATH_INFO: `/v3/projects/197e55aa210c42abbf12e585a968f8c5` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:04.408654 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a9b788-c927-427d-9e2f-61e86095ae6d None admin] RBAC: Authorizing `identity:get_project(project_id=197e55aa210c42abbf12e585a968f8c5)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:04.410299 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a9b788-c927-427d-9e2f-61e86095ae6d None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:04.412177 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 210/445] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 6 19:59:04 2022] GET /identity/v3/projects/197e55aa210c42abbf12e585a968f8c5 => generated 298 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:04.413108 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a9b788-c927-427d-9e2f-61e86095ae6d None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:04.473807 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:04.509337 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:04.540204 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f96ac6-bb3f-4e81-abca-0cbde10a5668 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:04.598788 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cbbf68-8bc3-4618-bc4d-a174142f4871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:04.599620 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cbbf68-8bc3-4618-bc4d-a174142f4871 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:04.600029 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cbbf68-8bc3-4618-bc4d-a174142f4871 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:04.600643 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cbbf68-8bc3-4618-bc4d-a174142f4871 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:04.601767 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cbbf68-8bc3-4618-bc4d-a174142f4871 None admin] RBAC: Adding query filter params (group.id=6a12d79ef6944c129aeb6eccf1643416, role.id=b02170460f054c259e825ab6e35f8537, scope.project.id=197e55aa210c42abbf12e585a968f8c5) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:04.602170 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cbbf68-8bc3-4618-bc4d-a174142f4871 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:04.603706 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a9b788-c927-427d-9e2f-61e86095ae6d None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:04.604457 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cbbf68-8bc3-4618-bc4d-a174142f4871 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:04.612045 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 236/446] 10.222.0.20 () {56 vars in 1376 bytes} [Tue Sep 6 19:59:04 2022] GET /identity/v3/role_assignments?group.id=6a12d79ef6944c129aeb6eccf1643416&scope.project.id=197e55aa210c42abbf12e585a968f8c5&role.id=b02170460f054c259e825ab6e35f8537 => generated 611 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:04.612936 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cbbf68-8bc3-4618-bc4d-a174142f4871 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:04.674770 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9c28eb-d4e4-4f5c-8b5a-a87ac1ea4d01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:04.675570 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9c28eb-d4e4-4f5c-8b5a-a87ac1ea4d01 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:04.676037 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9c28eb-d4e4-4f5c-8b5a-a87ac1ea4d01 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:04.676432 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9c28eb-d4e4-4f5c-8b5a-a87ac1ea4d01 None admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:04.677487 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9c28eb-d4e4-4f5c-8b5a-a87ac1ea4d01 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:04.678356 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9c28eb-d4e4-4f5c-8b5a-a87ac1ea4d01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:04.680011 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9c28eb-d4e4-4f5c-8b5a-a87ac1ea4d01 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:04.693054 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 211/447] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 6 19:59:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:04.696180 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9c28eb-d4e4-4f5c-8b5a-a87ac1ea4d01 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:04.819447 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cbbf68-8bc3-4618-bc4d-a174142f4871 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:04.873322 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:04.879937 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9c28eb-d4e4-4f5c-8b5a-a87ac1ea4d01 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:04.915073 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116637f7-5636-45b5-8933-be323ee2439a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:04.916653 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116637f7-5636-45b5-8933-be323ee2439a None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:04.917018 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116637f7-5636-45b5-8933-be323ee2439a None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:04.919771 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116637f7-5636-45b5-8933-be323ee2439a None admin] PATH_INFO: `/v3/groups/6a12d79ef6944c129aeb6eccf1643416` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:04.919771 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116637f7-5636-45b5-8933-be323ee2439a None admin] RBAC: Authorizing `identity:get_group(group_id=6a12d79ef6944c129aeb6eccf1643416)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:04.923401 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116637f7-5636-45b5-8933-be323ee2439a None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:04.926250 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 237/448] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Sep 6 19:59:04 2022] GET /identity/v3/groups/6a12d79ef6944c129aeb6eccf1643416 => generated 226 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:04.928591 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116637f7-5636-45b5-8933-be323ee2439a None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:04.952568 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd2caef-dec4-4979-8c7b-e194e1100a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:04.953820 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd2caef-dec4-4979-8c7b-e194e1100a0c None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:04.953820 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd2caef-dec4-4979-8c7b-e194e1100a0c None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:04.954088 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd2caef-dec4-4979-8c7b-e194e1100a0c None admin] PATH_INFO: `/v3/groups/6a12d79ef6944c129aeb6eccf1643416` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:04.955350 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd2caef-dec4-4979-8c7b-e194e1100a0c None admin] RBAC: Authorizing `identity:get_group(group_id=6a12d79ef6944c129aeb6eccf1643416)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:04.957070 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd2caef-dec4-4979-8c7b-e194e1100a0c None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:04.961718 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 212/449] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Sep 6 19:59:04 2022] GET /identity/v3/groups/6a12d79ef6944c129aeb6eccf1643416 => generated 226 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:04.961718 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd2caef-dec4-4979-8c7b-e194e1100a0c None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:04.970822 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:05.082997 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116637f7-5636-45b5-8933-be323ee2439a None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:05.127157 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd2caef-dec4-4979-8c7b-e194e1100a0c None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:05.133018 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3ce1b7-1c4b-46e4-a8ef-c8b156beb5c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:05.134179 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3ce1b7-1c4b-46e4-a8ef-c8b156beb5c1 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:05.134681 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3ce1b7-1c4b-46e4-a8ef-c8b156beb5c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:05.135189 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3ce1b7-1c4b-46e4-a8ef-c8b156beb5c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:05.166891 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f69d7f-24a1-4325-827c-099392cdd559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:05.167766 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f69d7f-24a1-4325-827c-099392cdd559 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:05.168054 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f69d7f-24a1-4325-827c-099392cdd559 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:05.168333 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f69d7f-24a1-4325-827c-099392cdd559 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:05.169124 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f69d7f-24a1-4325-827c-099392cdd559 None admin] RBAC: Adding query filter params (group.id=6a12d79ef6944c129aeb6eccf1643416, role.id=b02170460f054c259e825ab6e35f8537, scope.project.id=22a00d6dfd004be4aa2f9727f02e69ab) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:05.169434 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f69d7f-24a1-4325-827c-099392cdd559 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:05.171129 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f69d7f-24a1-4325-827c-099392cdd559 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:05.178757 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 213/450] 10.222.0.20 () {56 vars in 1376 bytes} [Tue Sep 6 19:59:04 2022] GET /identity/v3/role_assignments?group.id=6a12d79ef6944c129aeb6eccf1643416&scope.project.id=22a00d6dfd004be4aa2f9727f02e69ab&role.id=b02170460f054c259e825ab6e35f8537 => generated 611 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:05.179907 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f69d7f-24a1-4325-827c-099392cdd559 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:05.237992 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:05.270261 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3ce1b7-1c4b-46e4-a8ef-c8b156beb5c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:05.271916 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3ce1b7-1c4b-46e4-a8ef-c8b156beb5c1 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:05.301332 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:05.373873 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f69d7f-24a1-4325-827c-099392cdd559 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:05.437261 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178bf014-9d7e-4557-9837-d2a6f0ea00b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:05.438232 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178bf014-9d7e-4557-9837-d2a6f0ea00b4 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:05.438589 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178bf014-9d7e-4557-9837-d2a6f0ea00b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:05.438921 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178bf014-9d7e-4557-9837-d2a6f0ea00b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:05.440042 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178bf014-9d7e-4557-9837-d2a6f0ea00b4 None admin] RBAC: Adding query filter params (group.id=6a12d79ef6944c129aeb6eccf1643416, role.id=4eea39e24b06431aa357053fe7907a3d, scope.project.id=197e55aa210c42abbf12e585a968f8c5) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:05.440496 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178bf014-9d7e-4557-9837-d2a6f0ea00b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:05.442603 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178bf014-9d7e-4557-9837-d2a6f0ea00b4 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:05.451898 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 214/451] 10.222.0.20 () {56 vars in 1376 bytes} [Tue Sep 6 19:59:05 2022] GET /identity/v3/role_assignments?group.id=6a12d79ef6944c129aeb6eccf1643416&scope.project.id=197e55aa210c42abbf12e585a968f8c5&role.id=4eea39e24b06431aa357053fe7907a3d => generated 611 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:05.476420 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 238/452] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 6 19:59:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 549 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 19:59:05.485326 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178bf014-9d7e-4557-9837-d2a6f0ea00b4 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:05.666063 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:05.678712 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178bf014-9d7e-4557-9837-d2a6f0ea00b4 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:05.728796 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fada9349-ca52-466b-99d6-abd05c944911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:05.729396 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fada9349-ca52-466b-99d6-abd05c944911 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:05.729730 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fada9349-ca52-466b-99d6-abd05c944911 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:05.730018 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fada9349-ca52-466b-99d6-abd05c944911 None admin] PATH_INFO: `/v3/projects/a630b7748ecd4893b8db1b6c1dc4efe9` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:05.731698 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fada9349-ca52-466b-99d6-abd05c944911 None admin] RBAC: Authorizing `identity:get_project(project_id=a630b7748ecd4893b8db1b6c1dc4efe9)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:05.733668 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fada9349-ca52-466b-99d6-abd05c944911 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:05.735870 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 215/453] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 6 19:59:05 2022] GET /identity/v3/projects/a630b7748ecd4893b8db1b6c1dc4efe9 => generated 344 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:05.743964 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3ce1b7-1c4b-46e4-a8ef-c8b156beb5c1 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:05.896207 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3ce1b7-1c4b-46e4-a8ef-c8b156beb5c1 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:05.921389 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:05.950779 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8137d9-a1f2-4e61-8ed1-a3670077ad9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:05.951489 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8137d9-a1f2-4e61-8ed1-a3670077ad9b None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:05.951813 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8137d9-a1f2-4e61-8ed1-a3670077ad9b None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:05.952153 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8137d9-a1f2-4e61-8ed1-a3670077ad9b None admin] PATH_INFO: `/v3/groups/c2db17bd65f74cfb8e12147856fca032` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:05.953490 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8137d9-a1f2-4e61-8ed1-a3670077ad9b None admin] RBAC: Authorizing `identity:get_group(group_id=c2db17bd65f74cfb8e12147856fca032)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:05.955716 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8137d9-a1f2-4e61-8ed1-a3670077ad9b None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:05.958416 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 239/454] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Sep 6 19:59:05 2022] GET /identity/v3/groups/c2db17bd65f74cfb8e12147856fca032 => generated 229 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:05.966780 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fada9349-ca52-466b-99d6-abd05c944911 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:06.119570 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fada9349-ca52-466b-99d6-abd05c944911 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:06.170502 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb2fa3b-2692-4ca4-ac93-46b010fa80b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:06.171290 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb2fa3b-2692-4ca4-ac93-46b010fa80b5 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:06.171717 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb2fa3b-2692-4ca4-ac93-46b010fa80b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:06.172122 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb2fa3b-2692-4ca4-ac93-46b010fa80b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:06.174001 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb2fa3b-2692-4ca4-ac93-46b010fa80b5 None admin] RBAC: Adding query filter params (group.id=c2db17bd65f74cfb8e12147856fca032, role.id=e0ab41d8ae22474688b5361bcc75e0f0, scope.project.id=a630b7748ecd4893b8db1b6c1dc4efe9) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:06.174519 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb2fa3b-2692-4ca4-ac93-46b010fa80b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:06.176922 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb2fa3b-2692-4ca4-ac93-46b010fa80b5 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:06.185567 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 216/455] 10.222.0.20 () {56 vars in 1376 bytes} [Tue Sep 6 19:59:05 2022] GET /identity/v3/role_assignments?group.id=c2db17bd65f74cfb8e12147856fca032&scope.project.id=a630b7748ecd4893b8db1b6c1dc4efe9&role.id=e0ab41d8ae22474688b5361bcc75e0f0 => generated 611 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:09.923541 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034618f6-392d-4691-98c2-065992f5eb67 None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:09.923541 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034618f6-392d-4691-98c2-065992f5eb67 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:09.923541 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034618f6-392d-4691-98c2-065992f5eb67 None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:09.931134 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 240/456] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 19:59:09 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 19:59:09.939294 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e624d992-9406-478b-a751-04576c919552 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:09.940074 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e624d992-9406-478b-a751-04576c919552 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:09.940266 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e624d992-9406-478b-a751-04576c919552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:09.999578 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-e624d992-9406-478b-a751-04576c919552 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 19:59:10.002278 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-e624d992-9406-478b-a751-04576c919552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 19:59:10.056403 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:10.186484 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 217/457] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 19:59:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 19:59:10.400776 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a22747-461f-4b07-9eaa-b57b0bd59e7f None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:10.400776 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a22747-461f-4b07-9eaa-b57b0bd59e7f None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:10.400776 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a22747-461f-4b07-9eaa-b57b0bd59e7f None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:10.401696 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 241/458] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 19:59:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 19:59:10.411190 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d7bab5-2585-447c-b5aa-cad79d7b9cff None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:10.411673 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d7bab5-2585-447c-b5aa-cad79d7b9cff None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:10.412160 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d7bab5-2585-447c-b5aa-cad79d7b9cff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:10.455692 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-29d7bab5-2585-447c-b5aa-cad79d7b9cff None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 19:59:10.457677 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-29d7bab5-2585-447c-b5aa-cad79d7b9cff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 19:59:10.513605 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4d054e-7e54-4757-886b-23a3cce104b9 None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:10.514557 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4d054e-7e54-4757-886b-23a3cce104b9 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:10.515329 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4d054e-7e54-4757-886b-23a3cce104b9 None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:10.519983 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 242/459] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 19:59:10 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 19:59:10.528114 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990c2433-c3e9-4d13-b164-9a33f345ce90 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:10.529195 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990c2433-c3e9-4d13-b164-9a33f345ce90 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:10.530003 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990c2433-c3e9-4d13-b164-9a33f345ce90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:10.567581 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-990c2433-c3e9-4d13-b164-9a33f345ce90 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 19:59:10.570343 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-990c2433-c3e9-4d13-b164-9a33f345ce90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 19:59:10.724176 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 218/460] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 19:59:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 19:59:10.726092 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d7bab5-2585-447c-b5aa-cad79d7b9cff None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:10.733496 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 243/461] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 19:59:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 19:59:10.902092 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:10.946250 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d7bab5-2585-447c-b5aa-cad79d7b9cff None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:11.027306 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392ba24d-5338-41bb-8870-c6c213138821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:11.028243 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392ba24d-5338-41bb-8870-c6c213138821 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:11.028724 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392ba24d-5338-41bb-8870-c6c213138821 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:11.029196 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392ba24d-5338-41bb-8870-c6c213138821 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:11.050371 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392ba24d-5338-41bb-8870-c6c213138821 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:11.053124 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392ba24d-5338-41bb-8870-c6c213138821 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:11.083480 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-392ba24d-5338-41bb-8870-c6c213138821 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 19:59:11.083480 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:11.083480 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:11.083480 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:11.083480 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:11.083480 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:11.083480 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:11.083480 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:11.083480 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:11.083480 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:11.083480 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:11.083480 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:11.083480 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 19:59:11.083480 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 19:59:11.083480 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 19:59:11.083480 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 19:59:11.085510 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:11.085510 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:11.085510 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:11.085510 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:11.085510 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:11.085510 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:11.085510 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:11.085510 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:11.085510 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:11.085510 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:11.085510 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:11.085510 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:11.085510 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:11.085510 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:11.085510 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:11.085510 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:11.085510 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 19:59:11.087479 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 19:59:11.087479 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 19:59:11.087479 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 19:59:11.090175 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 219/462] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 19:59:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 365 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:59:11.097398 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990c2433-c3e9-4d13-b164-9a33f345ce90 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:11.288986 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392ba24d-5338-41bb-8870-c6c213138821 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:11.315349 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990c2433-c3e9-4d13-b164-9a33f345ce90 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:11.346096 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:11.384248 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75dcc03-9e50-486f-a192-92e8058b8d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:11.385557 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75dcc03-9e50-486f-a192-92e8058b8d94 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:11.386297 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75dcc03-9e50-486f-a192-92e8058b8d94 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:11.386946 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75dcc03-9e50-486f-a192-92e8058b8d94 None admin] PATH_INFO: `/v3/domains` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:11.388310 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75dcc03-9e50-486f-a192-92e8058b8d94 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:11.389309 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75dcc03-9e50-486f-a192-92e8058b8d94 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:11.392608 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75dcc03-9e50-486f-a192-92e8058b8d94 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:11.414649 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 244/463] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 6 19:59:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:11.423862 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75dcc03-9e50-486f-a192-92e8058b8d94 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:11.504058 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392ba24d-5338-41bb-8870-c6c213138821 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:11.550839 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e58ffe6-6cb8-4351-b7fe-6614eea63079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:11.551596 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e58ffe6-6cb8-4351-b7fe-6614eea63079 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:11.552292 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e58ffe6-6cb8-4351-b7fe-6614eea63079 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:11.552577 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e58ffe6-6cb8-4351-b7fe-6614eea63079 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:11.562930 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:11.570301 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e58ffe6-6cb8-4351-b7fe-6614eea63079 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:11.572536 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e58ffe6-6cb8-4351-b7fe-6614eea63079 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:11.582934 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75dcc03-9e50-486f-a192-92e8058b8d94 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:11.592765 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-7e58ffe6-6cb8-4351-b7fe-6614eea63079 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 19:59:11.592765 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:11.592765 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:11.592765 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:11.592765 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:11.592765 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:11.592765 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:11.592765 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:11.592765 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:11.592765 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:11.592765 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:11.592765 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:11.592765 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 19:59:11.592765 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 19:59:11.592765 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 19:59:11.592765 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 19:59:11.593908 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:11.593908 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:11.593908 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:11.593908 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:11.593908 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:11.593908 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:11.593908 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:11.593908 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:11.593908 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:11.593908 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:11.593908 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:11.593908 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:11.593908 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:11.593908 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:11.593908 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:11.593908 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:11.593908 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 19:59:11.595519 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 19:59:11.595519 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 19:59:11.595519 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 19:59:11.595789 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 220/464] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 19:59:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:59:11.596721 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e58ffe6-6cb8-4351-b7fe-6614eea63079 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:11.635632 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5176a977-78ab-442c-b498-ead6c1bd7ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:11.636442 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5176a977-78ab-442c-b498-ead6c1bd7ba7 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:11.636878 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5176a977-78ab-442c-b498-ead6c1bd7ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:11.637346 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5176a977-78ab-442c-b498-ead6c1bd7ba7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:11.654179 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5176a977-78ab-442c-b498-ead6c1bd7ba7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:11.656140 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5176a977-78ab-442c-b498-ead6c1bd7ba7 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:11.678702 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-5176a977-78ab-442c-b498-ead6c1bd7ba7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 19:59:11.678702 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:11.678702 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:11.678702 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:11.678702 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:11.678702 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:11.678702 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:11.678702 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:11.678702 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:11.678702 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:11.678702 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:11.678702 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:11.678702 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 19:59:11.678702 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 19:59:11.678702 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 19:59:11.678702 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 19:59:11.680468 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:11.680468 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:11.680468 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:11.680468 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:11.680468 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:11.680468 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:11.680468 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:11.680468 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:11.680468 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:11.680468 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:11.680468 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:11.680468 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:11.680468 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:11.680468 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:11.680468 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:11.680468 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:11.680468 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 19:59:11.682660 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 19:59:11.682660 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 19:59:11.682660 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 19:59:11.682660 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 245/465] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 19:59:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:59:11.682660 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354673f6-4779-4da0-ad4d-56cdcb33c998 None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:11.682660 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354673f6-4779-4da0-ad4d-56cdcb33c998 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:11.682660 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354673f6-4779-4da0-ad4d-56cdcb33c998 None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:11.690438 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 246/466] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 19:59:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 19:59:11.690438 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354673f6-4779-4da0-ad4d-56cdcb33c998 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:11.739364 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:11.775962 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e58ffe6-6cb8-4351-b7fe-6614eea63079 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:11.846067 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac3ceaa-be0b-4503-bced-bd8a7f80b0ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:11.846804 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac3ceaa-be0b-4503-bced-bd8a7f80b0ec None admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:11.847188 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac3ceaa-be0b-4503-bced-bd8a7f80b0ec None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:11.847571 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac3ceaa-be0b-4503-bced-bd8a7f80b0ec None admin] PATH_INFO: `/v3/users` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:11.848642 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac3ceaa-be0b-4503-bced-bd8a7f80b0ec None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:11.851466 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac3ceaa-be0b-4503-bced-bd8a7f80b0ec None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:11.893840 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354673f6-4779-4da0-ad4d-56cdcb33c998 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:11.905660 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 221/467] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Sep 6 19:59:11 2022] POST /identity/v3/users => generated 248 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 19:59:11.905660 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac3ceaa-be0b-4503-bced-bd8a7f80b0ec None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:11.967766 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76881ad6-b15b-4b81-a89c-491d41071aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:11.969097 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76881ad6-b15b-4b81-a89c-491d41071aec None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:11.969676 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76881ad6-b15b-4b81-a89c-491d41071aec None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:11.970323 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76881ad6-b15b-4b81-a89c-491d41071aec None admin] PATH_INFO: `/v3/domains` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:11.971751 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76881ad6-b15b-4b81-a89c-491d41071aec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:11.972426 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76881ad6-b15b-4b81-a89c-491d41071aec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:11.975278 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76881ad6-b15b-4b81-a89c-491d41071aec None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:12.001840 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 247/468] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 6 19:59:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:12.004550 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905876a0-923d-406b-ab70-a041ca43d068 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:12.005145 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905876a0-923d-406b-ab70-a041ca43d068 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:12.005767 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905876a0-923d-406b-ab70-a041ca43d068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:12.013104 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:12.044321 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-905876a0-923d-406b-ab70-a041ca43d068 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 19:59:12.046933 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-905876a0-923d-406b-ab70-a041ca43d068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 19:59:12.115760 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac3ceaa-be0b-4503-bced-bd8a7f80b0ec None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:12.170103 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a032deb1-3000-4a63-aa5b-1d26c8740410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:12.170767 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a032deb1-3000-4a63-aa5b-1d26c8740410 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:12.171046 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a032deb1-3000-4a63-aa5b-1d26c8740410 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:12.171356 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a032deb1-3000-4a63-aa5b-1d26c8740410 None admin] PATH_INFO: `/v3/domains` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:12.172139 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a032deb1-3000-4a63-aa5b-1d26c8740410 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:12.172473 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a032deb1-3000-4a63-aa5b-1d26c8740410 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:12.174722 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a032deb1-3000-4a63-aa5b-1d26c8740410 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:12.191319 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 248/469] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 19:59:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 19:59:12.192720 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905876a0-923d-406b-ab70-a041ca43d068 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:12.195292 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 222/470] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 6 19:59:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:12.196881 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9653b4-7ce1-4f97-99a1-a3c2debfe77a None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:12.197309 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9653b4-7ce1-4f97-99a1-a3c2debfe77a None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:12.197924 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9653b4-7ce1-4f97-99a1-a3c2debfe77a None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:12.201784 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 223/471] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 19:59:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 19:59:12.203909 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9653b4-7ce1-4f97-99a1-a3c2debfe77a None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:12.242302 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:12.414328 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905876a0-923d-406b-ab70-a041ca43d068 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:12.416720 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9653b4-7ce1-4f97-99a1-a3c2debfe77a None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:12.434512 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:12.481193 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07653cbe-3014-48b2-b448-bbdb3237ee3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:12.482446 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07653cbe-3014-48b2-b448-bbdb3237ee3c None admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:12.483024 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07653cbe-3014-48b2-b448-bbdb3237ee3c None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:12.483593 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07653cbe-3014-48b2-b448-bbdb3237ee3c None admin] PATH_INFO: `/v3/users` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:12.485065 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07653cbe-3014-48b2-b448-bbdb3237ee3c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:12.488287 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07653cbe-3014-48b2-b448-bbdb3237ee3c None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:12.494907 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea50f91-2b34-47cd-92d9-6492d19fdbe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:12.496061 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea50f91-2b34-47cd-92d9-6492d19fdbe6 None admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:12.496653 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea50f91-2b34-47cd-92d9-6492d19fdbe6 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:12.497221 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea50f91-2b34-47cd-92d9-6492d19fdbe6 None admin] PATH_INFO: `/v3/users` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:12.498581 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea50f91-2b34-47cd-92d9-6492d19fdbe6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:12.502201 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea50f91-2b34-47cd-92d9-6492d19fdbe6 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:12.523962 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 249/472] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Sep 6 19:59:12 2022] POST /identity/v3/users => generated 248 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 19:59:12.527640 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf99707d-f84d-4168-9298-f3fe1f0b02f1 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:12.528309 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf99707d-f84d-4168-9298-f3fe1f0b02f1 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:12.528754 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf99707d-f84d-4168-9298-f3fe1f0b02f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:12.535804 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 224/473] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Sep 6 19:59:12 2022] POST /identity/v3/users => generated 246 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 19:59:12.537185 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea50f91-2b34-47cd-92d9-6492d19fdbe6 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:12.598629 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-bf99707d-f84d-4168-9298-f3fe1f0b02f1 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 19:59:12.603816 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-bf99707d-f84d-4168-9298-f3fe1f0b02f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 19:59:12.852707 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea50f91-2b34-47cd-92d9-6492d19fdbe6 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:12.868593 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 250/474] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 19:59:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 19:59:12.926529 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878c1095-da2f-4abe-a1a8-9c57d3f5bf22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:12.927202 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878c1095-da2f-4abe-a1a8-9c57d3f5bf22 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:12.927485 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878c1095-da2f-4abe-a1a8-9c57d3f5bf22 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:12.927762 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878c1095-da2f-4abe-a1a8-9c57d3f5bf22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:12.941434 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878c1095-da2f-4abe-a1a8-9c57d3f5bf22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:12.943151 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878c1095-da2f-4abe-a1a8-9c57d3f5bf22 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:12.960359 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-878c1095-da2f-4abe-a1a8-9c57d3f5bf22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 19:59:12.960359 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:12.960359 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:12.960359 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:12.960359 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:12.960359 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:12.960359 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:12.960359 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:12.960359 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:12.960359 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:12.960359 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:12.960359 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:12.960359 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 19:59:12.960359 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 19:59:12.960359 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 19:59:12.960359 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 19:59:12.962760 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:12.962760 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:12.962760 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:12.962760 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:12.962760 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:12.962760 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:12.962760 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:12.962760 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:12.962760 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:12.962760 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:12.962760 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:12.962760 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:12.962760 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:12.962760 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:12.962760 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:12.962760 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:12.962760 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 19:59:12.964582 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 19:59:12.964582 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 19:59:12.964582 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 19:59:12.964582 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 225/475] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 19:59:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 425 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:59:12.975810 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf99707d-f84d-4168-9298-f3fe1f0b02f1 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:12.981411 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:13.164836 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf99707d-f84d-4168-9298-f3fe1f0b02f1 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:13.241990 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f62e621-5bad-4230-9829-8b33e06439af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:13.244287 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f62e621-5bad-4230-9829-8b33e06439af None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:13.245046 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f62e621-5bad-4230-9829-8b33e06439af None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:13.245771 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f62e621-5bad-4230-9829-8b33e06439af None admin] PATH_INFO: `/v3/domains` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:13.247503 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f62e621-5bad-4230-9829-8b33e06439af None admin] RBAC: Adding query filter params (name=Default) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:13.248377 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f62e621-5bad-4230-9829-8b33e06439af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:13.251875 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f62e621-5bad-4230-9829-8b33e06439af None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:13.285679 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 251/476] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 6 19:59:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:13.291285 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878c1095-da2f-4abe-a1a8-9c57d3f5bf22 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:13.324242 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:13.418405 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f62e621-5bad-4230-9829-8b33e06439af None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:13.460786 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878c1095-da2f-4abe-a1a8-9c57d3f5bf22 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:13.501921 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c499059b-746c-43e4-afe0-f10f61c86b5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:13.502556 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c499059b-746c-43e4-afe0-f10f61c86b5c None admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:13.502840 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c499059b-746c-43e4-afe0-f10f61c86b5c None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:13.503120 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c499059b-746c-43e4-afe0-f10f61c86b5c None admin] PATH_INFO: `/v3/users` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:13.504058 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c499059b-746c-43e4-afe0-f10f61c86b5c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:13.505741 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c499059b-746c-43e4-afe0-f10f61c86b5c None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:13.548648 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 226/477] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Sep 6 19:59:13 2022] POST /identity/v3/users => generated 249 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 19:59:13.582260 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:13.682916 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f62e621-5bad-4230-9829-8b33e06439af None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:13.761260 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4b53c2-37e7-4f63-9e8c-2397cfc9df9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:13.762874 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4b53c2-37e7-4f63-9e8c-2397cfc9df9f None admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:13.763668 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4b53c2-37e7-4f63-9e8c-2397cfc9df9f None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:13.764717 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4b53c2-37e7-4f63-9e8c-2397cfc9df9f None admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:13.766373 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4b53c2-37e7-4f63-9e8c-2397cfc9df9f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:13.769351 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4b53c2-37e7-4f63-9e8c-2397cfc9df9f None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:13.789075 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.sql.core [None req-3f4b53c2-37e7-4f63-9e8c-2397cfc9df9f None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 06 19:59:13.789075 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 06 19:59:13.789075 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [parameters: {'id': '8e2110e0d05041ccaf41a259d9bdcef9', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 06 19:59:13.789075 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81772) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 06 19:59:13.824118 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-3f4b53c2-37e7-4f63-9e8c-2397cfc9df9f None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 06 19:59:13.824118 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:13.824118 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 06 19:59:13.824118 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 06 19:59:13.824118 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 06 19:59:13.824118 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 06 19:59:13.824118 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 06 19:59:13.824118 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application result = self._query(query) Sep 06 19:59:13.824118 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 06 19:59:13.824118 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application conn.query(q) Sep 06 19:59:13.824118 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 06 19:59:13.824118 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 06 19:59:13.824118 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 06 19:59:13.824118 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application result.read() Sep 06 19:59:13.824118 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 06 19:59:13.825881 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 06 19:59:13.825881 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 06 19:59:13.825881 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 06 19:59:13.825881 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 06 19:59:13.825881 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 06 19:59:13.825881 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 06 19:59:13.825881 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 06 19:59:13.825881 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 06 19:59:13.825881 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 19:59:13.825881 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 06 19:59:13.825881 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 19:59:13.825881 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:13.825881 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 06 19:59:13.825881 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 06 19:59:13.825881 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 06 19:59:13.825881 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return ref.to_dict() Sep 06 19:59:13.825881 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 06 19:59:13.825881 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application next(self.gen) Sep 06 19:59:13.827675 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 06 19:59:13.827675 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application yield resource Sep 06 19:59:13.827675 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 06 19:59:13.827675 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application next(self.gen) Sep 06 19:59:13.827675 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 06 19:59:13.827675 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application self.session.rollback() Sep 06 19:59:13.827675 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 06 19:59:13.827675 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application self.force_reraise() Sep 06 19:59:13.827675 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 06 19:59:13.827675 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise self.value Sep 06 19:59:13.827675 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 06 19:59:13.827675 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 06 19:59:13.827675 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 06 19:59:13.827675 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application session.commit() Sep 06 19:59:13.827675 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 06 19:59:13.827675 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 06 19:59:13.827675 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 06 19:59:13.829453 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application self._prepare_impl() Sep 06 19:59:13.829453 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 06 19:59:13.829453 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application self.session.flush() Sep 06 19:59:13.829453 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 06 19:59:13.829453 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application self._flush(objects) Sep 06 19:59:13.829453 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 06 19:59:13.829453 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 06 19:59:13.829453 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 06 19:59:13.829453 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application compat.raise_( Sep 06 19:59:13.829453 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 06 19:59:13.829453 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception Sep 06 19:59:13.829453 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 06 19:59:13.829453 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application flush_context.execute() Sep 06 19:59:13.829453 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 06 19:59:13.829453 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rec.execute(self) Sep 06 19:59:13.829453 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 06 19:59:13.829453 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 06 19:59:13.833821 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 06 19:59:13.833821 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 06 19:59:13.833821 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 06 19:59:13.833821 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 06 19:59:13.833821 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 06 19:59:13.833821 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 06 19:59:13.833821 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 06 19:59:13.833821 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 06 19:59:13.833821 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 06 19:59:13.833821 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 06 19:59:13.833821 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 06 19:59:13.833821 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 06 19:59:13.833821 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 06 19:59:13.833821 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 06 19:59:13.833821 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 06 19:59:13.833821 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception Sep 06 19:59:13.839236 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 06 19:59:13.839236 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 06 19:59:13.839236 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 06 19:59:13.839236 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 06 19:59:13.839236 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 06 19:59:13.839236 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application result = self._query(query) Sep 06 19:59:13.839236 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 06 19:59:13.839236 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application conn.query(q) Sep 06 19:59:13.839236 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 06 19:59:13.839236 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 06 19:59:13.839236 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 06 19:59:13.839236 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application result.read() Sep 06 19:59:13.839236 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 06 19:59:13.839236 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 06 19:59:13.839236 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 06 19:59:13.839236 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 06 19:59:13.843364 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 06 19:59:13.843364 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 06 19:59:13.843364 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 06 19:59:13.843364 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 06 19:59:13.843364 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 06 19:59:13.843364 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 06 19:59:13.843364 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [parameters: {'id': '8e2110e0d05041ccaf41a259d9bdcef9', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 06 19:59:13.843364 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 06 19:59:13.843364 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 19:59:13.843364 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 19:59:13.843364 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 19:59:13.843364 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:13.843364 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:13.843364 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:13.843364 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:13.843364 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:13.847278 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:13.847278 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:13.847278 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:13.847278 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:13.847278 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:13.847278 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:13.847278 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 06 19:59:13.847278 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 06 19:59:13.847278 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:13.847278 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:13.847278 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 06 19:59:13.847278 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 06 19:59:13.847278 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 06 19:59:13.847278 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 06 19:59:13.847278 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 06 19:59:13.847278 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 19:59:13.850422 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 252/478] 10.222.0.20 () {60 vars in 1131 bytes} [Tue Sep 6 19:59:13 2022] POST /identity/v3/roles => generated 144 bytes in 425 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:59:13.854811 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c499059b-746c-43e4-afe0-f10f61c86b5c None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:14.006647 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:14.089764 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c499059b-746c-43e4-afe0-f10f61c86b5c None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:14.177485 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15374a0e-d39c-48da-9ec8-4f0102fa4e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:14.178775 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15374a0e-d39c-48da-9ec8-4f0102fa4e38 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:14.179262 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15374a0e-d39c-48da-9ec8-4f0102fa4e38 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:14.179747 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15374a0e-d39c-48da-9ec8-4f0102fa4e38 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:14.215429 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15374a0e-d39c-48da-9ec8-4f0102fa4e38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 19:59:14.226227 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-15374a0e-d39c-48da-9ec8-4f0102fa4e38 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 19:59:14.226227 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:14.226227 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:14.226227 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:14.226227 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:14.226227 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:14.226227 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:14.226227 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:14.226227 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:14.226227 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:14.226227 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:14.226227 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:14.226227 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 19:59:14.226227 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 19:59:14.226227 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 19:59:14.226227 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 19:59:14.227951 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 19:59:14.227951 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 19:59:14.227951 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 19:59:14.227951 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 19:59:14.227951 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:14.227951 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:14.227951 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:14.227951 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:14.227951 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:14.227951 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:14.227951 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:14.227951 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:14.227951 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:14.227951 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:14.227951 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:14.227951 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:14.230508 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:14.230508 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:14.230508 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:14.230508 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:14.230508 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 19:59:14.230508 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 19:59:14.230508 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 19:59:14.230508 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 19:59:14.230508 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 19:59:14.230508 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 19:59:14.230508 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 19:59:14.230508 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 19:59:14.230508 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 227/479] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Sep 6 19:59:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 374 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:59:14.238925 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4b53c2-37e7-4f63-9e8c-2397cfc9df9f None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:14.456351 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4b53c2-37e7-4f63-9e8c-2397cfc9df9f None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:14.508961 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:14.525900 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ff1fb8-1725-485c-bb46-9683b33f85c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:14.527454 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ff1fb8-1725-485c-bb46-9683b33f85c0 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:14.528174 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ff1fb8-1725-485c-bb46-9683b33f85c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:14.529036 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ff1fb8-1725-485c-bb46-9683b33f85c0 None admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:14.530435 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ff1fb8-1725-485c-bb46-9683b33f85c0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:14.531103 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ff1fb8-1725-485c-bb46-9683b33f85c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:14.534566 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ff1fb8-1725-485c-bb46-9683b33f85c0 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:14.552003 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 253/480] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 6 19:59:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:15.811878 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620c37dd-ccc2-469b-94d9-72d82619e828 None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:15.812771 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620c37dd-ccc2-469b-94d9-72d82619e828 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:15.813195 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620c37dd-ccc2-469b-94d9-72d82619e828 None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:15.815354 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 228/481] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 19:59:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 19:59:15.822105 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e819459f-c022-46e2-a1ca-d7f64fdc5dab None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:15.823096 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e819459f-c022-46e2-a1ca-d7f64fdc5dab None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:15.823592 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e819459f-c022-46e2-a1ca-d7f64fdc5dab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:15.858415 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-e819459f-c022-46e2-a1ca-d7f64fdc5dab None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 19:59:15.860631 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-e819459f-c022-46e2-a1ca-d7f64fdc5dab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 19:59:15.862642 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9457c362-f048-4183-8935-f04336c33395 None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:15.862926 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9457c362-f048-4183-8935-f04336c33395 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:15.863206 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9457c362-f048-4183-8935-f04336c33395 None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:15.865118 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 229/482] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 19:59:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 19:59:15.871601 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413c0ebb-bcff-40a1-922a-42c9b5890b46 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:15.871961 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413c0ebb-bcff-40a1-922a-42c9b5890b46 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:15.872336 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413c0ebb-bcff-40a1-922a-42c9b5890b46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:15.917029 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-413c0ebb-bcff-40a1-922a-42c9b5890b46 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 19:59:15.918582 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-413c0ebb-bcff-40a1-922a-42c9b5890b46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 19:59:16.004418 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 254/483] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 19:59:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 19:59:16.093859 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 230/484] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 19:59:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 19:59:16.328209 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e819459f-c022-46e2-a1ca-d7f64fdc5dab None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:16.450685 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413c0ebb-bcff-40a1-922a-42c9b5890b46 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:16.542056 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e819459f-c022-46e2-a1ca-d7f64fdc5dab None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:16.582078 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:16.602807 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:16.618883 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3898950-9755-45e4-b263-39be9c3722bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:16.619962 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3898950-9755-45e4-b263-39be9c3722bb None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:16.620452 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3898950-9755-45e4-b263-39be9c3722bb None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:16.620916 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3898950-9755-45e4-b263-39be9c3722bb None admin] PATH_INFO: `/v3/roles/service` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:16.651328 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3898950-9755-45e4-b263-39be9c3722bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 19:59:16.661110 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-b3898950-9755-45e4-b263-39be9c3722bb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 19:59:16.661110 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:16.661110 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:16.661110 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:16.661110 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:16.661110 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:16.661110 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:16.661110 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:16.661110 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:16.661110 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:16.661110 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:16.661110 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:16.661110 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 19:59:16.661110 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 19:59:16.661110 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 19:59:16.661110 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 19:59:16.662145 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 19:59:16.662145 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 19:59:16.662145 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 19:59:16.662145 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 19:59:16.662145 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:16.662145 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:16.662145 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:16.662145 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:16.662145 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:16.662145 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:16.662145 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:16.662145 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:16.662145 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:16.662145 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:16.662145 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:16.662145 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:16.662947 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:16.662947 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:16.662947 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:16.662947 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:16.662947 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 19:59:16.662947 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 19:59:16.662947 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 19:59:16.662947 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 19:59:16.662947 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 19:59:16.662947 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 19:59:16.662947 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 19:59:16.662947 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 19:59:16.663713 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 255/485] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Sep 6 19:59:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:59:16.667044 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f65668-c6ce-43bc-962b-d59399cef29f None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:16.667444 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f65668-c6ce-43bc-962b-d59399cef29f None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:16.667840 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f65668-c6ce-43bc-962b-d59399cef29f None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:16.671205 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 256/486] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 19:59:16 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 19:59:16.672811 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f65668-c6ce-43bc-962b-d59399cef29f None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:16.713412 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413c0ebb-bcff-40a1-922a-42c9b5890b46 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:16.790879 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49be699-b6b9-4703-b634-2bc32398b3bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:16.791976 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49be699-b6b9-4703-b634-2bc32398b3bd None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:16.792449 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49be699-b6b9-4703-b634-2bc32398b3bd None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:16.794061 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49be699-b6b9-4703-b634-2bc32398b3bd None admin] PATH_INFO: `/v3/roles/service` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:16.815951 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49be699-b6b9-4703-b634-2bc32398b3bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 19:59:16.830986 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-d49be699-b6b9-4703-b634-2bc32398b3bd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 19:59:16.830986 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:16.830986 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:16.830986 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:16.830986 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:16.830986 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:16.830986 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:16.830986 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:16.830986 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:16.830986 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:16.830986 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:16.830986 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:16.830986 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 19:59:16.830986 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 19:59:16.830986 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 19:59:16.830986 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 19:59:16.833470 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 19:59:16.833470 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 19:59:16.833470 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 19:59:16.833470 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 19:59:16.833470 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:16.833470 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:16.833470 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:16.833470 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:16.833470 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:16.833470 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:16.833470 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:16.833470 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:16.833470 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:16.833470 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:16.833470 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:16.833470 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:16.835880 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:16.835880 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:16.835880 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:16.835880 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:16.835880 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 19:59:16.835880 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 19:59:16.835880 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 19:59:16.835880 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 19:59:16.835880 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 19:59:16.835880 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 19:59:16.835880 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 19:59:16.835880 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 19:59:16.835880 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 231/487] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Sep 6 19:59:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 385 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:59:16.844146 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60a82e6-6bd9-4dec-b69f-1d512ddc3440 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:16.844146 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60a82e6-6bd9-4dec-b69f-1d512ddc3440 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:16.844146 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60a82e6-6bd9-4dec-b69f-1d512ddc3440 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:16.871359 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-f60a82e6-6bd9-4dec-b69f-1d512ddc3440 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 19:59:16.872797 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-f60a82e6-6bd9-4dec-b69f-1d512ddc3440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 19:59:16.900193 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f65668-c6ce-43bc-962b-d59399cef29f None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:16.962234 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 232/488] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 19:59:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 19:59:16.963134 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60a82e6-6bd9-4dec-b69f-1d512ddc3440 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:16.990936 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0dcc06-373f-4c17-b8e4-7eb5d4b5f448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:16.991978 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0dcc06-373f-4c17-b8e4-7eb5d4b5f448 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:16.992473 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0dcc06-373f-4c17-b8e4-7eb5d4b5f448 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:16.992952 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0dcc06-373f-4c17-b8e4-7eb5d4b5f448 None admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:16.994557 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0dcc06-373f-4c17-b8e4-7eb5d4b5f448 None admin] RBAC: Adding query filter params (name=service) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:16.995108 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0dcc06-373f-4c17-b8e4-7eb5d4b5f448 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:16.997945 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0dcc06-373f-4c17-b8e4-7eb5d4b5f448 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:17.031812 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:17.035995 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 257/489] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Sep 6 19:59:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 364 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:17.043528 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0dcc06-373f-4c17-b8e4-7eb5d4b5f448 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:17.103267 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60a82e6-6bd9-4dec-b69f-1d512ddc3440 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:17.154047 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323d28b2-da86-4c96-9a6c-f559154e4b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:17.154605 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323d28b2-da86-4c96-9a6c-f559154e4b21 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:17.155499 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323d28b2-da86-4c96-9a6c-f559154e4b21 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:17.155499 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323d28b2-da86-4c96-9a6c-f559154e4b21 None admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:17.156369 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323d28b2-da86-4c96-9a6c-f559154e4b21 None admin] RBAC: Adding query filter params (name=service) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:17.157144 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323d28b2-da86-4c96-9a6c-f559154e4b21 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:17.158731 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323d28b2-da86-4c96-9a6c-f559154e4b21 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:17.169271 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 233/490] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Sep 6 19:59:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:17.181340 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:17.182064 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323d28b2-da86-4c96-9a6c-f559154e4b21 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:17.305450 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0dcc06-373f-4c17-b8e4-7eb5d4b5f448 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:17.330983 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323d28b2-da86-4c96-9a6c-f559154e4b21 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:17.366144 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:17.397544 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd37ed0-5861-48f2-9335-b4aa062c02d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:17.398336 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd37ed0-5861-48f2-9335-b4aa062c02d5 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:17.398696 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd37ed0-5861-48f2-9335-b4aa062c02d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:17.398960 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd37ed0-5861-48f2-9335-b4aa062c02d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:17.402846 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d104ab7-ab54-4be1-978e-c616c93d4b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:17.403613 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d104ab7-ab54-4be1-978e-c616c93d4b9c None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:17.403930 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d104ab7-ab54-4be1-978e-c616c93d4b9c None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:17.404204 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d104ab7-ab54-4be1-978e-c616c93d4b9c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:17.428140 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd37ed0-5861-48f2-9335-b4aa062c02d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:17.429829 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd37ed0-5861-48f2-9335-b4aa062c02d5 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:17.430568 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d104ab7-ab54-4be1-978e-c616c93d4b9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:17.432468 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d104ab7-ab54-4be1-978e-c616c93d4b9c None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:17.449022 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-acd37ed0-5861-48f2-9335-b4aa062c02d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 19:59:17.449022 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:17.449022 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:17.449022 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:17.449022 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:17.449022 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:17.449022 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:17.449022 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:17.449022 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:17.449022 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:17.449022 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:17.449022 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:17.449022 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 19:59:17.449022 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 19:59:17.449022 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 19:59:17.449022 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 19:59:17.450820 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:17.450820 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:17.450820 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:17.450820 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:17.450820 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:17.450820 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:17.450820 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:17.450820 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:17.450820 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:17.450820 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:17.450820 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:17.450820 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:17.450820 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:17.450820 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:17.450820 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:17.450820 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:17.450820 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 19:59:17.452406 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 19:59:17.452406 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 19:59:17.452406 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 19:59:17.452406 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 234/491] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 19:59:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:59:17.457446 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd37ed0-5861-48f2-9335-b4aa062c02d5 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:17.472383 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-4d104ab7-ab54-4be1-978e-c616c93d4b9c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 19:59:17.472383 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:17.472383 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:17.472383 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:17.472383 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:17.472383 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:17.472383 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:17.472383 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:17.472383 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:17.472383 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:17.472383 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:17.472383 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:17.472383 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 19:59:17.472383 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 19:59:17.472383 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 19:59:17.472383 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 19:59:17.473889 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:17.473889 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:17.473889 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:17.473889 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:17.473889 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:17.473889 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:17.473889 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:17.473889 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:17.473889 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:17.473889 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:17.473889 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:17.473889 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:17.473889 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:17.473889 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:17.473889 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:17.473889 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:17.473889 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 19:59:17.475411 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 19:59:17.475411 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 19:59:17.475411 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 19:59:17.478516 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 258/492] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 19:59:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 436 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:59:17.484036 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d104ab7-ab54-4be1-978e-c616c93d4b9c None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:17.684145 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:17.698967 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd37ed0-5861-48f2-9335-b4aa062c02d5 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:17.707608 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d104ab7-ab54-4be1-978e-c616c93d4b9c None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:17.748251 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aee383a-792a-480d-a9d5-324d04b0e486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:17.749730 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aee383a-792a-480d-a9d5-324d04b0e486 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:17.750174 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aee383a-792a-480d-a9d5-324d04b0e486 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:17.750661 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aee383a-792a-480d-a9d5-324d04b0e486 None admin] PATH_INFO: `/v3/domains` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:17.752623 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aee383a-792a-480d-a9d5-324d04b0e486 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:17.753167 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aee383a-792a-480d-a9d5-324d04b0e486 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:17.756488 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aee383a-792a-480d-a9d5-324d04b0e486 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:17.767392 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d01ac5f-1725-4dc2-8644-838e16eaec9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:17.768023 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d01ac5f-1725-4dc2-8644-838e16eaec9d None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:17.768300 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d01ac5f-1725-4dc2-8644-838e16eaec9d None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:17.768619 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d01ac5f-1725-4dc2-8644-838e16eaec9d None admin] PATH_INFO: `/v3/domains` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:17.769297 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d01ac5f-1725-4dc2-8644-838e16eaec9d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:17.769606 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d01ac5f-1725-4dc2-8644-838e16eaec9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:17.771083 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d01ac5f-1725-4dc2-8644-838e16eaec9d None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:17.783747 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 235/493] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 6 19:59:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:17.785664 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aee383a-792a-480d-a9d5-324d04b0e486 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:17.796577 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 259/494] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 6 19:59:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:17.800275 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29147ea9-d19e-4034-9153-ec3858631ca2 None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:17.800873 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29147ea9-d19e-4034-9153-ec3858631ca2 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:17.801422 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29147ea9-d19e-4034-9153-ec3858631ca2 None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:17.809038 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 260/495] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 19:59:17 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 19:59:17.809038 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29147ea9-d19e-4034-9153-ec3858631ca2 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:17.829230 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:17.973093 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aee383a-792a-480d-a9d5-324d04b0e486 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:17.985860 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29147ea9-d19e-4034-9153-ec3858631ca2 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:18.037201 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a81312c-d7da-4b65-8c28-44940433a437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:18.038792 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a81312c-d7da-4b65-8c28-44940433a437 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:18.039595 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a81312c-d7da-4b65-8c28-44940433a437 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:18.040120 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a81312c-d7da-4b65-8c28-44940433a437 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:18.046884 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facb2791-66d4-4c9b-a0d8-05c137ebcffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:18.047889 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facb2791-66d4-4c9b-a0d8-05c137ebcffc None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:18.048359 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facb2791-66d4-4c9b-a0d8-05c137ebcffc None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:18.049323 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facb2791-66d4-4c9b-a0d8-05c137ebcffc None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:18.059515 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a81312c-d7da-4b65-8c28-44940433a437 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 19:59:18.061870 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facb2791-66d4-4c9b-a0d8-05c137ebcffc None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:18.064693 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facb2791-66d4-4c9b-a0d8-05c137ebcffc None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:18.068937 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-1a81312c-d7da-4b65-8c28-44940433a437 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 19:59:18.068937 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:18.068937 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:18.068937 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:18.068937 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:18.068937 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:18.068937 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:18.068937 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:18.068937 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:18.068937 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:18.068937 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:18.068937 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:18.068937 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 19:59:18.068937 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 19:59:18.068937 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 19:59:18.068937 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 19:59:18.069931 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 19:59:18.069931 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 19:59:18.069931 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 19:59:18.069931 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 19:59:18.069931 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:18.069931 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:18.069931 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:18.069931 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:18.069931 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:18.069931 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:18.069931 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:18.069931 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:18.069931 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:18.069931 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:18.069931 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:18.069931 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:18.070857 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:18.070857 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:18.070857 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:18.070857 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:18.070857 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 19:59:18.070857 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 19:59:18.070857 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 19:59:18.070857 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 19:59:18.070857 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 19:59:18.070857 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 19:59:18.070857 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 19:59:18.070857 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 19:59:18.070857 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 236/496] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Sep 6 19:59:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:59:18.072087 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a81312c-d7da-4b65-8c28-44940433a437 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:18.089216 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-facb2791-66d4-4c9b-a0d8-05c137ebcffc None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 06 19:59:18.089216 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:18.089216 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:18.089216 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:18.089216 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:18.089216 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:18.089216 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:18.089216 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:18.089216 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:18.089216 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:18.089216 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:18.089216 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:18.089216 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 19:59:18.089216 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 19:59:18.089216 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 19:59:18.089216 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 19:59:18.090310 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:18.090310 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:18.090310 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 19:59:18.090310 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 19:59:18.090310 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 19:59:18.090310 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 19:59:18.090310 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:18.090310 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:18.090310 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:18.090310 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:18.090310 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:18.090310 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:18.090310 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:18.090310 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:18.090310 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:18.090310 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:18.090310 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:18.090310 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:18.091361 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:18.091361 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:18.091361 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 19:59:18.091361 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 19:59:18.091361 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 19:59:18.091361 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 19:59:18.091361 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 19:59:18.091361 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 19:59:18.091361 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 06 19:59:18.091361 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 19:59:18.094052 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 261/497] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 6 19:59:17 2022] GET /identity/v3/users/cinder => generated 84 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:59:18.096509 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad3adfd-c798-407c-91a3-f102cfcb56d7 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:18.097010 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad3adfd-c798-407c-91a3-f102cfcb56d7 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:18.097502 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad3adfd-c798-407c-91a3-f102cfcb56d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:18.120145 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-2ad3adfd-c798-407c-91a3-f102cfcb56d7 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 19:59:18.121754 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-2ad3adfd-c798-407c-91a3-f102cfcb56d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 19:59:18.133076 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:18.202024 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:18.228423 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a81312c-d7da-4b65-8c28-44940433a437 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:18.233191 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 262/498] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 19:59:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 19:59:18.234072 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad3adfd-c798-407c-91a3-f102cfcb56d7 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:18.324117 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bae89ad-b911-4979-be86-edc017b4c2ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:18.326292 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bae89ad-b911-4979-be86-edc017b4c2ba None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:18.327977 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bae89ad-b911-4979-be86-edc017b4c2ba None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:18.329266 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bae89ad-b911-4979-be86-edc017b4c2ba None admin] PATH_INFO: `/v3/users/glance` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:18.344288 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bae89ad-b911-4979-be86-edc017b4c2ba None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:18.346428 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bae89ad-b911-4979-be86-edc017b4c2ba None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:18.363799 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-6bae89ad-b911-4979-be86-edc017b4c2ba None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 06 19:59:18.363799 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:18.363799 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:18.363799 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:18.363799 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:18.363799 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:18.363799 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:18.363799 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:18.363799 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:18.363799 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:18.363799 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:18.363799 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:18.363799 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 19:59:18.363799 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 19:59:18.363799 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 19:59:18.363799 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 19:59:18.365455 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:18.365455 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:18.365455 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 19:59:18.365455 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 19:59:18.365455 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 19:59:18.365455 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 19:59:18.365455 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:18.365455 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:18.365455 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:18.365455 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:18.365455 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:18.365455 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:18.365455 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:18.365455 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:18.365455 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:18.365455 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:18.365455 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:18.365455 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:18.367145 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:18.367145 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:18.367145 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 19:59:18.367145 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 19:59:18.367145 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 19:59:18.367145 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 19:59:18.367145 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 19:59:18.367145 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 19:59:18.367145 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 06 19:59:18.367145 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 19:59:18.370086 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 237/499] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 6 19:59:18 2022] GET /identity/v3/users/glance => generated 84 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:59:18.371090 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bae89ad-b911-4979-be86-edc017b4c2ba None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:18.429767 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad3adfd-c798-407c-91a3-f102cfcb56d7 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:18.508255 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f524efe-a9ab-4692-a457-e2a7f6b9bc32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:18.510550 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f524efe-a9ab-4692-a457-e2a7f6b9bc32 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:18.511028 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f524efe-a9ab-4692-a457-e2a7f6b9bc32 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:18.511491 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f524efe-a9ab-4692-a457-e2a7f6b9bc32 None admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:18.512687 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f524efe-a9ab-4692-a457-e2a7f6b9bc32 None admin] RBAC: Adding query filter params (name=service) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:18.513223 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f524efe-a9ab-4692-a457-e2a7f6b9bc32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:18.516076 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f524efe-a9ab-4692-a457-e2a7f6b9bc32 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:18.532470 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 263/500] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Sep 6 19:59:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:18.534547 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f524efe-a9ab-4692-a457-e2a7f6b9bc32 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:18.541285 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:18.569335 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bae89ad-b911-4979-be86-edc017b4c2ba None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:18.637434 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3abbfe-eb5d-4666-8b21-0aaa4ae3fe19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:18.638365 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3abbfe-eb5d-4666-8b21-0aaa4ae3fe19 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:18.638819 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3abbfe-eb5d-4666-8b21-0aaa4ae3fe19 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:18.639289 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3abbfe-eb5d-4666-8b21-0aaa4ae3fe19 None admin] PATH_INFO: `/v3/users` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:18.640958 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3abbfe-eb5d-4666-8b21-0aaa4ae3fe19 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:18.641491 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3abbfe-eb5d-4666-8b21-0aaa4ae3fe19 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:18.644797 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3abbfe-eb5d-4666-8b21-0aaa4ae3fe19 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:18.696381 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f524efe-a9ab-4692-a457-e2a7f6b9bc32 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:18.714603 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 238/501] 10.222.0.20 () {56 vars in 1137 bytes} [Tue Sep 6 19:59:18 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:18.716830 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498dfdf3-4769-455f-bb3b-4933547f97e1 None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:18.717308 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498dfdf3-4769-455f-bb3b-4933547f97e1 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:18.717828 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498dfdf3-4769-455f-bb3b-4933547f97e1 None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:18.723301 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 239/502] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 19:59:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 19:59:18.723301 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498dfdf3-4769-455f-bb3b-4933547f97e1 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:18.727457 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:18.758280 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4319c528-bacf-4ed4-bd66-33bda486408a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:18.759348 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4319c528-bacf-4ed4-bd66-33bda486408a None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:18.759806 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4319c528-bacf-4ed4-bd66-33bda486408a None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:18.760285 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4319c528-bacf-4ed4-bd66-33bda486408a None admin] PATH_INFO: `/v3/users` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:18.761559 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4319c528-bacf-4ed4-bd66-33bda486408a None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:18.762277 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4319c528-bacf-4ed4-bd66-33bda486408a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:18.764215 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4319c528-bacf-4ed4-bd66-33bda486408a None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:18.818657 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 264/503] 10.222.0.20 () {56 vars in 1137 bytes} [Tue Sep 6 19:59:18 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:18.819570 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4319c528-bacf-4ed4-bd66-33bda486408a None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:18.887872 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498dfdf3-4769-455f-bb3b-4933547f97e1 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:18.936737 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:18.940597 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98800be9-161d-4477-b4a5-64f0dba761fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:18.941344 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98800be9-161d-4477-b4a5-64f0dba761fd None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:18.941774 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98800be9-161d-4477-b4a5-64f0dba761fd None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:18.942117 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98800be9-161d-4477-b4a5-64f0dba761fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:18.956471 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98800be9-161d-4477-b4a5-64f0dba761fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:18.958261 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98800be9-161d-4477-b4a5-64f0dba761fd None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:18.977562 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-98800be9-161d-4477-b4a5-64f0dba761fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 19:59:18.977562 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:18.977562 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:18.977562 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:18.977562 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:18.977562 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:18.977562 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:18.977562 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:18.977562 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:18.977562 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:18.977562 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:18.977562 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:18.977562 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 19:59:18.977562 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 19:59:18.977562 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 19:59:18.977562 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 19:59:18.979448 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:18.979448 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:18.979448 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:18.979448 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:18.979448 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:18.979448 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:18.979448 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:18.979448 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:18.979448 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:18.979448 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:18.979448 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:18.979448 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:18.979448 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:18.979448 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:18.979448 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:18.979448 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:18.979448 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 19:59:18.981092 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 19:59:18.981092 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 19:59:18.981092 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 19:59:18.981092 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 240/504] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 19:59:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:59:18.984079 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98800be9-161d-4477-b4a5-64f0dba761fd None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:18.988410 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4319c528-bacf-4ed4-bd66-33bda486408a None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:19.039423 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bf4b4a-1cc6-4425-a52c-fd6b6e584e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:19.040143 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bf4b4a-1cc6-4425-a52c-fd6b6e584e56 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:19.040557 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bf4b4a-1cc6-4425-a52c-fd6b6e584e56 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:19.040920 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bf4b4a-1cc6-4425-a52c-fd6b6e584e56 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:19.054351 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bf4b4a-1cc6-4425-a52c-fd6b6e584e56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 19:59:19.059835 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-67bf4b4a-1cc6-4425-a52c-fd6b6e584e56 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 19:59:19.059835 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:19.059835 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:19.059835 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:19.059835 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:19.059835 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:19.059835 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:19.059835 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:19.059835 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:19.059835 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:19.059835 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:19.059835 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:19.059835 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 19:59:19.059835 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 19:59:19.059835 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 19:59:19.059835 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 19:59:19.061604 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 19:59:19.061604 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 19:59:19.061604 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 19:59:19.061604 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 19:59:19.061604 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:19.061604 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:19.061604 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:19.061604 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:19.061604 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:19.061604 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:19.061604 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:19.061604 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:19.061604 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:19.061604 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:19.061604 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:19.061604 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:19.063216 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:19.063216 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:19.063216 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:19.063216 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:19.063216 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 19:59:19.063216 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 19:59:19.063216 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 19:59:19.063216 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 19:59:19.063216 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 19:59:19.063216 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 19:59:19.063216 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 19:59:19.063216 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 19:59:19.063216 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 265/505] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Sep 6 19:59:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:59:19.064543 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f999f4d-b815-43e0-bebe-44d5f786af00 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:19.064543 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f999f4d-b815-43e0-bebe-44d5f786af00 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:19.064850 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f999f4d-b815-43e0-bebe-44d5f786af00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:19.081122 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:19.090220 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-1f999f4d-b815-43e0-bebe-44d5f786af00 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 19:59:19.091472 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-1f999f4d-b815-43e0-bebe-44d5f786af00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 19:59:19.155079 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98800be9-161d-4477-b4a5-64f0dba761fd None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:19.197075 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 266/506] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 19:59:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 19:59:19.198585 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f999f4d-b815-43e0-bebe-44d5f786af00 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:19.217091 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958b2454-7935-4a78-8e27-10c3b7dcef54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:19.218180 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958b2454-7935-4a78-8e27-10c3b7dcef54 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:19.218683 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958b2454-7935-4a78-8e27-10c3b7dcef54 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:19.219148 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958b2454-7935-4a78-8e27-10c3b7dcef54 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:19.293777 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:19.358430 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958b2454-7935-4a78-8e27-10c3b7dcef54 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:19.361721 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958b2454-7935-4a78-8e27-10c3b7dcef54 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:19.369190 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f999f4d-b815-43e0-bebe-44d5f786af00 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:19.424172 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef6a62d-1e3a-4a35-b259-351a71112065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:19.425083 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef6a62d-1e3a-4a35-b259-351a71112065 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:19.425371 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef6a62d-1e3a-4a35-b259-351a71112065 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:19.425730 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef6a62d-1e3a-4a35-b259-351a71112065 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:19.474645 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:19.532106 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef6a62d-1e3a-4a35-b259-351a71112065 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:19.534934 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef6a62d-1e3a-4a35-b259-351a71112065 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:19.558230 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 241/507] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 6 19:59:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 578 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 19:59:19.559608 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958b2454-7935-4a78-8e27-10c3b7dcef54 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:19.615310 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:19.700149 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958b2454-7935-4a78-8e27-10c3b7dcef54 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:19.700710 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 267/508] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 6 19:59:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 19:59:19.701775 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef6a62d-1e3a-4a35-b259-351a71112065 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:19.773381 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f529b2a-3389-48a0-8fed-785240af1544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:19.774337 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f529b2a-3389-48a0-8fed-785240af1544 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:19.774781 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f529b2a-3389-48a0-8fed-785240af1544 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:19.775256 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f529b2a-3389-48a0-8fed-785240af1544 None admin] PATH_INFO: `/v3/domains` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:19.776139 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f529b2a-3389-48a0-8fed-785240af1544 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:19.776658 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f529b2a-3389-48a0-8fed-785240af1544 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:19.779130 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f529b2a-3389-48a0-8fed-785240af1544 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:19.796256 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 242/509] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 6 19:59:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:19.797187 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f529b2a-3389-48a0-8fed-785240af1544 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:19.817495 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:19.877276 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef6a62d-1e3a-4a35-b259-351a71112065 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:19.931277 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac845c7-6ff5-442e-b8be-8df3ebe7deb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:19.932148 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac845c7-6ff5-442e-b8be-8df3ebe7deb4 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:19.932576 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac845c7-6ff5-442e-b8be-8df3ebe7deb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:19.933096 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac845c7-6ff5-442e-b8be-8df3ebe7deb4 None admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:19.933867 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac845c7-6ff5-442e-b8be-8df3ebe7deb4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:19.934178 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac845c7-6ff5-442e-b8be-8df3ebe7deb4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:19.935816 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac845c7-6ff5-442e-b8be-8df3ebe7deb4 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:19.948761 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 268/510] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Sep 6 19:59:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:19.949763 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac845c7-6ff5-442e-b8be-8df3ebe7deb4 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:19.975702 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:19.977668 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f529b2a-3389-48a0-8fed-785240af1544 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:20.031671 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94841068-51d7-4db4-846e-48b465179131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:20.032653 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94841068-51d7-4db4-846e-48b465179131 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:20.033708 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94841068-51d7-4db4-846e-48b465179131 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:20.034190 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94841068-51d7-4db4-846e-48b465179131 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:20.049543 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94841068-51d7-4db4-846e-48b465179131 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:20.051681 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94841068-51d7-4db4-846e-48b465179131 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:20.077653 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-94841068-51d7-4db4-846e-48b465179131 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 19:59:20.077653 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:20.077653 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:20.077653 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:20.077653 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:20.077653 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:20.077653 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:20.077653 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:20.077653 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:20.077653 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:20.077653 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:20.077653 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:20.077653 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 19:59:20.077653 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 19:59:20.077653 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 19:59:20.077653 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 19:59:20.079536 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:20.079536 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:20.079536 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:20.079536 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:20.079536 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:20.079536 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:20.079536 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:20.079536 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:20.079536 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:20.079536 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:20.079536 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:20.079536 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:20.079536 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:20.079536 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:20.079536 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:20.079536 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:20.079536 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 19:59:20.082056 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 19:59:20.082056 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 19:59:20.082056 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 19:59:20.082056 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 243/511] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 19:59:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:59:20.082056 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94841068-51d7-4db4-846e-48b465179131 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:20.098248 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac845c7-6ff5-442e-b8be-8df3ebe7deb4 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:20.138637 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bd9f37-af25-40c7-80ec-c5aed6b33e07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:20.139286 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bd9f37-af25-40c7-80ec-c5aed6b33e07 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:20.139568 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bd9f37-af25-40c7-80ec-c5aed6b33e07 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:20.140102 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bd9f37-af25-40c7-80ec-c5aed6b33e07 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:20.152846 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bd9f37-af25-40c7-80ec-c5aed6b33e07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:20.154475 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bd9f37-af25-40c7-80ec-c5aed6b33e07 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:20.170416 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-52bd9f37-af25-40c7-80ec-c5aed6b33e07 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 19:59:20.170416 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:20.170416 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:20.170416 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:20.170416 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:20.170416 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:20.170416 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:20.170416 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:20.170416 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:20.170416 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:20.170416 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:20.170416 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:20.170416 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 19:59:20.170416 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 19:59:20.170416 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 19:59:20.170416 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 19:59:20.172226 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:20.172226 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:20.172226 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:20.172226 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:20.172226 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:20.172226 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:20.172226 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:20.172226 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:20.172226 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:20.172226 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:20.172226 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:20.172226 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:20.172226 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:20.172226 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:20.172226 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:20.172226 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:20.172226 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 19:59:20.174419 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 19:59:20.174419 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 19:59:20.174419 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 19:59:20.174743 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 269/512] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 19:59:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:59:20.176005 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bd9f37-af25-40c7-80ec-c5aed6b33e07 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:20.204925 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:20.269555 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:20.270981 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94841068-51d7-4db4-846e-48b465179131 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:20.334998 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea73b0ac-b47f-4ab4-8a33-771f695fcd29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:20.335785 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea73b0ac-b47f-4ab4-8a33-771f695fcd29 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:20.336149 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea73b0ac-b47f-4ab4-8a33-771f695fcd29 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:20.336507 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea73b0ac-b47f-4ab4-8a33-771f695fcd29 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:20.337431 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bd9f37-af25-40c7-80ec-c5aed6b33e07 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:20.351217 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea73b0ac-b47f-4ab4-8a33-771f695fcd29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:20.352886 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea73b0ac-b47f-4ab4-8a33-771f695fcd29 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:20.371288 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-ea73b0ac-b47f-4ab4-8a33-771f695fcd29 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 19:59:20.371288 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:20.371288 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:20.371288 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:20.371288 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:20.371288 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:20.371288 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:20.371288 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:20.371288 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:20.371288 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:20.371288 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:20.371288 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:20.371288 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 19:59:20.371288 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 19:59:20.371288 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 19:59:20.371288 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 19:59:20.373050 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:20.373050 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:20.373050 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:20.373050 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:20.373050 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:20.373050 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:20.373050 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:20.373050 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:20.373050 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:20.373050 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:20.373050 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:20.373050 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:20.373050 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:20.373050 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:20.373050 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:20.373050 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:20.373050 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 19:59:20.374762 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 19:59:20.374762 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 19:59:20.374762 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 19:59:20.374762 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 244/513] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 19:59:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:59:20.375186 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea73b0ac-b47f-4ab4-8a33-771f695fcd29 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:20.386223 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66edf350-7e35-4c96-8795-163984323491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:20.387221 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66edf350-7e35-4c96-8795-163984323491 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:20.387665 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66edf350-7e35-4c96-8795-163984323491 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:20.388114 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66edf350-7e35-4c96-8795-163984323491 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:20.402420 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66edf350-7e35-4c96-8795-163984323491 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:20.405009 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66edf350-7e35-4c96-8795-163984323491 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:20.427702 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-66edf350-7e35-4c96-8795-163984323491 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 06 19:59:20.427702 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:20.427702 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:20.427702 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:20.427702 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:20.427702 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:20.427702 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:20.427702 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:20.427702 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:20.427702 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:20.427702 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:20.427702 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:20.427702 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 19:59:20.427702 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 19:59:20.427702 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 19:59:20.427702 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 19:59:20.429541 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:20.429541 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:20.429541 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 19:59:20.429541 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 19:59:20.429541 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 19:59:20.429541 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 19:59:20.429541 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:20.429541 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:20.429541 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:20.429541 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:20.429541 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:20.429541 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:20.429541 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:20.429541 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:20.429541 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:20.429541 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:20.429541 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:20.429541 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:20.430616 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:20.430616 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:20.430616 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 19:59:20.430616 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 19:59:20.430616 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 19:59:20.430616 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 19:59:20.430616 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 19:59:20.430616 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 19:59:20.430616 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 06 19:59:20.430616 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 19:59:20.430616 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 270/514] 10.222.0.20 () {56 vars in 1093 bytes} [Tue Sep 6 19:59:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:59:20.431184 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66edf350-7e35-4c96-8795-163984323491 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:20.524750 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea73b0ac-b47f-4ab4-8a33-771f695fcd29 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:20.564015 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:20.568992 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66edf350-7e35-4c96-8795-163984323491 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:20.587032 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f538188-fd0d-4c6e-9098-2535a9c96bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:20.587974 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f538188-fd0d-4c6e-9098-2535a9c96bf7 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:20.588384 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f538188-fd0d-4c6e-9098-2535a9c96bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:20.588785 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f538188-fd0d-4c6e-9098-2535a9c96bf7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:20.605734 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f538188-fd0d-4c6e-9098-2535a9c96bf7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:20.607159 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f538188-fd0d-4c6e-9098-2535a9c96bf7 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:20.608901 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:20.626636 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e22ec02-dfe6-467c-85a1-7f068f2f7126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:20.627960 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e22ec02-dfe6-467c-85a1-7f068f2f7126 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:20.628334 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e22ec02-dfe6-467c-85a1-7f068f2f7126 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:20.628694 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e22ec02-dfe6-467c-85a1-7f068f2f7126 None admin] PATH_INFO: `/v3/domains` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:20.629484 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e22ec02-dfe6-467c-85a1-7f068f2f7126 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:20.629922 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e22ec02-dfe6-467c-85a1-7f068f2f7126 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:20.630619 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-5f538188-fd0d-4c6e-9098-2535a9c96bf7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 19:59:20.630619 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:20.630619 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:20.630619 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:20.630619 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:20.630619 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:20.630619 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:20.630619 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:20.630619 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:20.630619 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:20.630619 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:20.630619 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:20.630619 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 19:59:20.630619 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 19:59:20.630619 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 19:59:20.630619 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 19:59:20.632280 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:20.632280 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:20.632280 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:20.632280 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:20.632280 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:20.632280 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:20.632280 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:20.632280 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:20.632280 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:20.632280 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:20.632280 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:20.632280 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:20.632280 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:20.632280 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:20.632280 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:20.632280 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:20.632280 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 19:59:20.634021 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 19:59:20.634021 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 19:59:20.634021 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 19:59:20.634021 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 245/515] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 19:59:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:59:20.634463 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e22ec02-dfe6-467c-85a1-7f068f2f7126 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:20.634591 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f538188-fd0d-4c6e-9098-2535a9c96bf7 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:20.661674 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 271/516] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 6 19:59:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:20.663807 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e22ec02-dfe6-467c-85a1-7f068f2f7126 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:20.810574 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f538188-fd0d-4c6e-9098-2535a9c96bf7 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:20.816800 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e22ec02-dfe6-467c-85a1-7f068f2f7126 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:20.859684 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81ad41c-d5f5-4b23-9a67-89ed5cbea5cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:20.860648 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81ad41c-d5f5-4b23-9a67-89ed5cbea5cf None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:20.861097 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81ad41c-d5f5-4b23-9a67-89ed5cbea5cf None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:20.862058 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81ad41c-d5f5-4b23-9a67-89ed5cbea5cf None admin] PATH_INFO: `/v3/domains` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:20.863024 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81ad41c-d5f5-4b23-9a67-89ed5cbea5cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:20.863523 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81ad41c-d5f5-4b23-9a67-89ed5cbea5cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:20.865967 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81ad41c-d5f5-4b23-9a67-89ed5cbea5cf None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:20.875089 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62749de-a3ed-4720-9c55-44341ff3f5fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:20.876046 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62749de-a3ed-4720-9c55-44341ff3f5fc None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:20.876514 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62749de-a3ed-4720-9c55-44341ff3f5fc None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:20.876975 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62749de-a3ed-4720-9c55-44341ff3f5fc None admin] PATH_INFO: `/v3/domains` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:20.878005 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62749de-a3ed-4720-9c55-44341ff3f5fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:20.878541 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62749de-a3ed-4720-9c55-44341ff3f5fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:20.880954 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62749de-a3ed-4720-9c55-44341ff3f5fc None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:20.886595 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 272/517] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 6 19:59:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:20.888092 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81ad41c-d5f5-4b23-9a67-89ed5cbea5cf None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:20.900490 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 246/518] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 6 19:59:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:20.901637 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62749de-a3ed-4720-9c55-44341ff3f5fc None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:20.941995 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:20.963307 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:21.038376 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81ad41c-d5f5-4b23-9a67-89ed5cbea5cf None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:21.055234 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62749de-a3ed-4720-9c55-44341ff3f5fc None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:21.103073 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb90639-2b26-49b9-88c8-2c3db3e7726f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:21.104056 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb90639-2b26-49b9-88c8-2c3db3e7726f None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:21.104056 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb90639-2b26-49b9-88c8-2c3db3e7726f None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:21.104352 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb90639-2b26-49b9-88c8-2c3db3e7726f None admin] PATH_INFO: `/v3/users` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:21.105122 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb90639-2b26-49b9-88c8-2c3db3e7726f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:21.105477 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb90639-2b26-49b9-88c8-2c3db3e7726f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:21.107072 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb90639-2b26-49b9-88c8-2c3db3e7726f None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:21.122142 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad64b49e-8b82-4085-a0f5-806d26554d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:21.123050 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad64b49e-8b82-4085-a0f5-806d26554d7f None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:21.123395 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad64b49e-8b82-4085-a0f5-806d26554d7f None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:21.123673 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad64b49e-8b82-4085-a0f5-806d26554d7f None admin] PATH_INFO: `/v3/domains` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:21.124384 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad64b49e-8b82-4085-a0f5-806d26554d7f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:21.124696 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad64b49e-8b82-4085-a0f5-806d26554d7f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:21.126278 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad64b49e-8b82-4085-a0f5-806d26554d7f None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:21.140849 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 247/519] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 6 19:59:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:21.141972 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 273/520] 10.222.0.20 () {56 vars in 1133 bytes} [Tue Sep 6 19:59:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:21.142151 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad64b49e-8b82-4085-a0f5-806d26554d7f None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:21.148531 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb90639-2b26-49b9-88c8-2c3db3e7726f None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:21.282848 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:21.296960 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad64b49e-8b82-4085-a0f5-806d26554d7f None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:21.311800 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb90639-2b26-49b9-88c8-2c3db3e7726f None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:21.315408 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:21.341435 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129a07e7-a804-4f8f-9c63-a2c232cac5e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:21.342196 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129a07e7-a804-4f8f-9c63-a2c232cac5e5 None admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:21.342485 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129a07e7-a804-4f8f-9c63-a2c232cac5e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:21.342768 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129a07e7-a804-4f8f-9c63-a2c232cac5e5 None admin] PATH_INFO: `/v3/users` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:21.343464 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129a07e7-a804-4f8f-9c63-a2c232cac5e5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:21.345057 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129a07e7-a804-4f8f-9c63-a2c232cac5e5 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:21.378940 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113ce94a-d0e6-41c4-b6ac-417a095a024e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:21.379956 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113ce94a-d0e6-41c4-b6ac-417a095a024e None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:21.380526 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113ce94a-d0e6-41c4-b6ac-417a095a024e None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:21.381068 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113ce94a-d0e6-41c4-b6ac-417a095a024e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:21.393946 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 248/521] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Sep 6 19:59:21 2022] POST /identity/v3/users => generated 247 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 19:59:21.393946 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129a07e7-a804-4f8f-9c63-a2c232cac5e5 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:21.403260 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113ce94a-d0e6-41c4-b6ac-417a095a024e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:21.403260 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113ce94a-d0e6-41c4-b6ac-417a095a024e None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:21.425870 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-113ce94a-d0e6-41c4-b6ac-417a095a024e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 19:59:21.425870 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:21.425870 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:21.425870 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:21.425870 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:21.425870 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:21.425870 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:21.425870 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:21.425870 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:21.425870 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:21.425870 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:21.425870 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:21.425870 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 19:59:21.425870 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 19:59:21.425870 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 19:59:21.427669 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 19:59:21.427669 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:21.427669 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:21.427669 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:21.427669 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:21.427669 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:21.427669 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:21.427669 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:21.427669 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:21.427669 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:21.427669 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:21.427669 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:21.427669 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:21.427669 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:21.427669 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:21.427669 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:21.427669 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:21.429663 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 19:59:21.429663 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 19:59:21.429663 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 19:59:21.429663 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 19:59:21.429663 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 19:59:21.429663 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 19:59:21.429663 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 19:59:21.429663 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 19:59:21.429663 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 274/522] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 6 19:59:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:59:21.432354 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113ce94a-d0e6-41c4-b6ac-417a095a024e None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:21.598109 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129a07e7-a804-4f8f-9c63-a2c232cac5e5 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:21.628078 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113ce94a-d0e6-41c4-b6ac-417a095a024e None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:21.651414 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8707a45d-1ea4-41a9-a901-3d7c62552669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:21.652364 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8707a45d-1ea4-41a9-a901-3d7c62552669 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:21.652833 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8707a45d-1ea4-41a9-a901-3d7c62552669 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:21.653320 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8707a45d-1ea4-41a9-a901-3d7c62552669 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:21.664496 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8707a45d-1ea4-41a9-a901-3d7c62552669 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:21.666774 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8707a45d-1ea4-41a9-a901-3d7c62552669 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:21.692192 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-8707a45d-1ea4-41a9-a901-3d7c62552669 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 19:59:21.692192 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:21.692192 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:21.692192 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:21.692192 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:21.692192 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:21.692192 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:21.692192 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:21.692192 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:21.692192 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:21.692192 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:21.692192 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:21.692192 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 19:59:21.692192 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 19:59:21.692192 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 19:59:21.694201 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 19:59:21.694201 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:21.694201 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:21.694201 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:21.694201 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:21.694201 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:21.694201 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:21.694201 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:21.694201 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:21.694201 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:21.694201 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:21.694201 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:21.694201 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:21.694201 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:21.694201 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:21.694201 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:21.694201 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:21.695950 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 19:59:21.695950 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 19:59:21.695950 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 19:59:21.695950 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 19:59:21.695950 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 19:59:21.695950 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 19:59:21.695950 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 19:59:21.695950 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 19:59:21.695950 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 249/523] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 6 19:59:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:59:21.695950 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8707a45d-1ea4-41a9-a901-3d7c62552669 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:21.701368 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94ef9d4-eecf-4b4d-9846-5021ae121056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:21.702373 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94ef9d4-eecf-4b4d-9846-5021ae121056 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:21.702784 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94ef9d4-eecf-4b4d-9846-5021ae121056 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:21.703242 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94ef9d4-eecf-4b4d-9846-5021ae121056 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:21.716684 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:21.719193 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94ef9d4-eecf-4b4d-9846-5021ae121056 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:21.721812 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94ef9d4-eecf-4b4d-9846-5021ae121056 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:21.747917 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-e94ef9d4-eecf-4b4d-9846-5021ae121056 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 06 19:59:21.747917 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:21.747917 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:21.747917 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:21.747917 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:21.747917 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:21.747917 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:21.747917 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:21.747917 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:21.747917 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:21.747917 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:21.747917 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:21.747917 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 19:59:21.747917 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 19:59:21.747917 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 19:59:21.747917 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 19:59:21.749231 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:21.749231 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:21.749231 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 19:59:21.749231 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 19:59:21.749231 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 19:59:21.749231 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 19:59:21.749231 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:21.749231 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:21.749231 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:21.749231 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:21.749231 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:21.749231 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:21.749231 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:21.749231 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:21.749231 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:21.749231 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:21.749231 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:21.749231 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:21.750338 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:21.750338 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:21.750338 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 19:59:21.750338 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 19:59:21.750338 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 19:59:21.750338 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 19:59:21.750338 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 19:59:21.750338 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 19:59:21.750338 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 06 19:59:21.750338 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 19:59:21.750338 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 275/524] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Sep 6 19:59:21 2022] GET /identity/v3/users/neutron => generated 85 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:59:21.751881 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94ef9d4-eecf-4b4d-9846-5021ae121056 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:21.804238 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:21.906246 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8707a45d-1ea4-41a9-a901-3d7c62552669 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:21.942965 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94ef9d4-eecf-4b4d-9846-5021ae121056 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:21.967628 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3276ce06-253b-47ae-8607-9284aab4f6b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:21.969223 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3276ce06-253b-47ae-8607-9284aab4f6b5 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:21.969680 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3276ce06-253b-47ae-8607-9284aab4f6b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:21.970084 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3276ce06-253b-47ae-8607-9284aab4f6b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:22.021423 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0297a6d6-4cce-44ca-a32f-3395987cb21c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:22.022451 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0297a6d6-4cce-44ca-a32f-3395987cb21c None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:22.023021 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0297a6d6-4cce-44ca-a32f-3395987cb21c None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:22.023380 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0297a6d6-4cce-44ca-a32f-3395987cb21c None admin] PATH_INFO: `/v3/projects` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:22.024639 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0297a6d6-4cce-44ca-a32f-3395987cb21c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:22.025269 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0297a6d6-4cce-44ca-a32f-3395987cb21c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:22.028054 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0297a6d6-4cce-44ca-a32f-3395987cb21c None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:22.072315 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 276/525] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Sep 6 19:59:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:22.072315 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0297a6d6-4cce-44ca-a32f-3395987cb21c None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:22.075164 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3276ce06-253b-47ae-8607-9284aab4f6b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:22.077245 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3276ce06-253b-47ae-8607-9284aab4f6b5 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:22.087767 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:22.234795 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 250/526] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 6 19:59:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 540 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 19:59:22.236900 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3276ce06-253b-47ae-8607-9284aab4f6b5 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:22.246083 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:22.280543 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0297a6d6-4cce-44ca-a32f-3395987cb21c None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:22.350824 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf6fae5-4269-4753-be23-9a9428a848ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:22.352350 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf6fae5-4269-4753-be23-9a9428a848ed None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:22.353092 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf6fae5-4269-4753-be23-9a9428a848ed None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:22.353843 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf6fae5-4269-4753-be23-9a9428a848ed None admin] PATH_INFO: `/v3/projects` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:22.355405 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf6fae5-4269-4753-be23-9a9428a848ed None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:22.356162 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf6fae5-4269-4753-be23-9a9428a848ed None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:22.359287 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf6fae5-4269-4753-be23-9a9428a848ed None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:22.384029 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 277/527] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Sep 6 19:59:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:22.385983 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf6fae5-4269-4753-be23-9a9428a848ed None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:22.418309 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3276ce06-253b-47ae-8607-9284aab4f6b5 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:22.429407 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:22.477063 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb2bb8f-af84-44fd-be5d-5e721a9a7b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:22.477956 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb2bb8f-af84-44fd-be5d-5e721a9a7b3a None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:22.478431 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb2bb8f-af84-44fd-be5d-5e721a9a7b3a None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:22.478906 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb2bb8f-af84-44fd-be5d-5e721a9a7b3a None admin] PATH_INFO: `/v3/users` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:22.479869 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb2bb8f-af84-44fd-be5d-5e721a9a7b3a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:22.480318 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb2bb8f-af84-44fd-be5d-5e721a9a7b3a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:22.482289 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb2bb8f-af84-44fd-be5d-5e721a9a7b3a None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:22.508184 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 251/528] 10.222.0.20 () {56 vars in 1139 bytes} [Tue Sep 6 19:59:22 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:22.508698 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb2bb8f-af84-44fd-be5d-5e721a9a7b3a None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:22.605467 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf6fae5-4269-4753-be23-9a9428a848ed None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:22.655473 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea3e92b-2480-4d7e-98d1-b43424324b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:22.656099 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea3e92b-2480-4d7e-98d1-b43424324b74 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:22.656380 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea3e92b-2480-4d7e-98d1-b43424324b74 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:22.656651 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea3e92b-2480-4d7e-98d1-b43424324b74 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:22.657412 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea3e92b-2480-4d7e-98d1-b43424324b74 None admin] RBAC: Adding query filter params (role.id=b950bf73c93e4f7ebed923ff14e39faa, scope.project.id=6aef43493bb1420a9e122313d8db7d5a, user.id=07af18f35ffc42a295278d5349e06381) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:22.657866 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea3e92b-2480-4d7e-98d1-b43424324b74 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:22.659767 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea3e92b-2480-4d7e-98d1-b43424324b74 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:22.677662 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 278/529] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Sep 6 19:59:22 2022] GET /identity/v3/role_assignments?user.id=07af18f35ffc42a295278d5349e06381&scope.project.id=6aef43493bb1420a9e122313d8db7d5a&role.id=b950bf73c93e4f7ebed923ff14e39faa => generated 259 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:22.677662 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea3e92b-2480-4d7e-98d1-b43424324b74 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:22.685129 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb2bb8f-af84-44fd-be5d-5e721a9a7b3a None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:22.700500 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:22.749701 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844c6bb5-4855-46b0-9897-8d8105a60b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:22.750805 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844c6bb5-4855-46b0-9897-8d8105a60b5f None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:22.751243 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844c6bb5-4855-46b0-9897-8d8105a60b5f None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:22.751676 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844c6bb5-4855-46b0-9897-8d8105a60b5f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:22.771136 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844c6bb5-4855-46b0-9897-8d8105a60b5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:22.773559 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844c6bb5-4855-46b0-9897-8d8105a60b5f None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:22.800995 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-844c6bb5-4855-46b0-9897-8d8105a60b5f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 19:59:22.800995 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:22.800995 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:22.800995 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:22.800995 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:22.800995 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:22.800995 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:22.800995 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:22.800995 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:22.800995 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:22.800995 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:22.800995 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:22.800995 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 19:59:22.800995 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 19:59:22.800995 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 19:59:22.800995 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 19:59:22.802581 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:22.802581 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:22.802581 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:22.802581 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:22.802581 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:22.802581 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:22.802581 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:22.802581 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:22.802581 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:22.802581 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:22.802581 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:22.802581 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:22.802581 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:22.802581 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:22.802581 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:22.802581 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:22.802581 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 19:59:22.806319 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 19:59:22.806319 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 19:59:22.806319 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 19:59:22.806319 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 252/530] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 19:59:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:59:22.806319 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844c6bb5-4855-46b0-9897-8d8105a60b5f None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:22.849999 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:22.894654 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea3e92b-2480-4d7e-98d1-b43424324b74 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:22.940856 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d59e2e-eb04-4d51-b383-65fd5cb0c591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:22.941668 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d59e2e-eb04-4d51-b383-65fd5cb0c591 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:22.942044 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d59e2e-eb04-4d51-b383-65fd5cb0c591 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:22.942593 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d59e2e-eb04-4d51-b383-65fd5cb0c591 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:22.943553 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d59e2e-eb04-4d51-b383-65fd5cb0c591 None admin] RBAC: Adding query filter params (role.id=b950bf73c93e4f7ebed923ff14e39faa, scope.project.id=6aef43493bb1420a9e122313d8db7d5a, user.id=df1bca993bc142908cc7cf85eabd2b24) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:22.943990 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d59e2e-eb04-4d51-b383-65fd5cb0c591 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:22.945650 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d59e2e-eb04-4d51-b383-65fd5cb0c591 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:22.955925 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 279/531] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Sep 6 19:59:22 2022] GET /identity/v3/role_assignments?user.id=df1bca993bc142908cc7cf85eabd2b24&scope.project.id=6aef43493bb1420a9e122313d8db7d5a&role.id=b950bf73c93e4f7ebed923ff14e39faa => generated 259 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:22.962755 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d59e2e-eb04-4d51-b383-65fd5cb0c591 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:22.963173 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844c6bb5-4855-46b0-9897-8d8105a60b5f None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:23.028597 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c527aeaa-456b-4b88-9043-2e96a8222b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:23.029316 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c527aeaa-456b-4b88-9043-2e96a8222b28 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:23.029737 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c527aeaa-456b-4b88-9043-2e96a8222b28 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:23.030095 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c527aeaa-456b-4b88-9043-2e96a8222b28 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:23.106822 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:23.187551 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d59e2e-eb04-4d51-b383-65fd5cb0c591 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:23.225832 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c527aeaa-456b-4b88-9043-2e96a8222b28 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:23.229215 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c527aeaa-456b-4b88-9043-2e96a8222b28 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:23.231302 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421d5aac-c015-4b8b-acc5-7d321bdba0dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:23.232002 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421d5aac-c015-4b8b-acc5-7d321bdba0dc None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:23.232509 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421d5aac-c015-4b8b-acc5-7d321bdba0dc None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:23.232808 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421d5aac-c015-4b8b-acc5-7d321bdba0dc None admin] PATH_INFO: `/v3/domains` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:23.233514 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421d5aac-c015-4b8b-acc5-7d321bdba0dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:23.233904 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421d5aac-c015-4b8b-acc5-7d321bdba0dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:23.235441 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421d5aac-c015-4b8b-acc5-7d321bdba0dc None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:23.251750 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 280/532] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 6 19:59:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:23.258472 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421d5aac-c015-4b8b-acc5-7d321bdba0dc None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:23.300340 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:23.449869 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421d5aac-c015-4b8b-acc5-7d321bdba0dc None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:23.516727 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:23.528307 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598cf6a7-9108-403e-a770-c6cc21fcbae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:23.529741 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598cf6a7-9108-403e-a770-c6cc21fcbae4 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:23.530256 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598cf6a7-9108-403e-a770-c6cc21fcbae4 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:23.530741 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598cf6a7-9108-403e-a770-c6cc21fcbae4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:23.549504 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598cf6a7-9108-403e-a770-c6cc21fcbae4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:23.553424 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598cf6a7-9108-403e-a770-c6cc21fcbae4 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:23.581793 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 253/533] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 6 19:59:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 776 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 19:59:23.588052 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c527aeaa-456b-4b88-9043-2e96a8222b28 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:23.594121 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-598cf6a7-9108-403e-a770-c6cc21fcbae4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 19:59:23.594121 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:23.594121 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:23.594121 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:23.594121 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:23.594121 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:23.594121 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:23.594121 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:23.594121 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:23.594121 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:23.594121 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:23.594121 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:23.594121 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 19:59:23.594121 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 19:59:23.594121 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 19:59:23.596731 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 19:59:23.596731 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:23.596731 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:23.596731 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:23.596731 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:23.596731 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:23.596731 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:23.596731 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:23.596731 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:23.596731 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:23.596731 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:23.596731 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:23.596731 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:23.596731 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:23.596731 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:23.596731 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:23.596731 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:23.599497 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 19:59:23.599497 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 19:59:23.599497 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 19:59:23.599497 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 19:59:23.599497 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 19:59:23.599497 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 19:59:23.599497 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 19:59:23.599497 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 19:59:23.599497 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 281/534] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 6 19:59:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:59:23.611432 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598cf6a7-9108-403e-a770-c6cc21fcbae4 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:23.821552 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c527aeaa-456b-4b88-9043-2e96a8222b28 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:23.847022 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:23.887312 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5809cab-930f-4afa-ab6d-40dc7d00944d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:23.888146 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5809cab-930f-4afa-ab6d-40dc7d00944d None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:23.888424 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5809cab-930f-4afa-ab6d-40dc7d00944d None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:23.888699 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5809cab-930f-4afa-ab6d-40dc7d00944d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:23.916992 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5809cab-930f-4afa-ab6d-40dc7d00944d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:23.927110 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5809cab-930f-4afa-ab6d-40dc7d00944d None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:23.931238 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598cf6a7-9108-403e-a770-c6cc21fcbae4 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:23.952244 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-c5809cab-930f-4afa-ab6d-40dc7d00944d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 19:59:23.952244 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:23.952244 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:23.952244 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:23.952244 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:23.952244 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:23.952244 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:23.952244 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:23.952244 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:23.952244 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:23.952244 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:23.952244 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:23.952244 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 19:59:23.952244 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 19:59:23.952244 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 19:59:23.952244 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 19:59:23.954484 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:23.954484 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:23.954484 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:23.954484 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:23.954484 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:23.954484 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:23.954484 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:23.954484 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:23.954484 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:23.954484 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:23.954484 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:23.954484 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:23.954484 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:23.954484 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:23.954484 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:23.954484 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:23.954484 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 19:59:23.956795 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 19:59:23.956795 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 19:59:23.956795 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 19:59:23.956795 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 254/535] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 19:59:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 367 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 19:59:23.966351 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5809cab-930f-4afa-ab6d-40dc7d00944d None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:24.062690 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeee102a-a9bf-4252-b5b5-f5b2f0426bb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:24.067497 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeee102a-a9bf-4252-b5b5-f5b2f0426bb1 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:24.068334 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeee102a-a9bf-4252-b5b5-f5b2f0426bb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:24.069452 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeee102a-a9bf-4252-b5b5-f5b2f0426bb1 None admin] PATH_INFO: `/v3/projects` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:24.073404 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeee102a-a9bf-4252-b5b5-f5b2f0426bb1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:24.074700 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeee102a-a9bf-4252-b5b5-f5b2f0426bb1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:24.079288 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeee102a-a9bf-4252-b5b5-f5b2f0426bb1 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:24.124814 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 282/536] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Sep 6 19:59:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 514 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:24.135628 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeee102a-a9bf-4252-b5b5-f5b2f0426bb1 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:24.193333 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5809cab-930f-4afa-ab6d-40dc7d00944d None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:24.289186 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7a5b9e-a3be-4783-b677-b83d204b9f11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:24.290643 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7a5b9e-a3be-4783-b677-b83d204b9f11 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:24.291145 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7a5b9e-a3be-4783-b677-b83d204b9f11 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:24.291590 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7a5b9e-a3be-4783-b677-b83d204b9f11 None admin] PATH_INFO: `/v3/domains` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:24.292441 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7a5b9e-a3be-4783-b677-b83d204b9f11 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:24.292924 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7a5b9e-a3be-4783-b677-b83d204b9f11 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:24.294813 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7a5b9e-a3be-4783-b677-b83d204b9f11 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:24.314187 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:24.324480 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 255/537] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 6 19:59:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:24.331750 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7a5b9e-a3be-4783-b677-b83d204b9f11 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:24.445282 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:24.477516 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeee102a-a9bf-4252-b5b5-f5b2f0426bb1 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:24.630880 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7a5b9e-a3be-4783-b677-b83d204b9f11 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:24.651532 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff75a515-deee-4a7f-8bd5-675c003a7a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:24.652515 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff75a515-deee-4a7f-8bd5-675c003a7a49 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:24.652992 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff75a515-deee-4a7f-8bd5-675c003a7a49 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:24.653461 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff75a515-deee-4a7f-8bd5-675c003a7a49 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:24.663115 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff75a515-deee-4a7f-8bd5-675c003a7a49 None admin] RBAC: Adding query filter params (role.id=b950bf73c93e4f7ebed923ff14e39faa, scope.project.id=6aef43493bb1420a9e122313d8db7d5a, user.id=5776cbf4f6714b2083c3ebdd6d98cc87) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:24.663677 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff75a515-deee-4a7f-8bd5-675c003a7a49 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:24.666392 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff75a515-deee-4a7f-8bd5-675c003a7a49 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:24.685491 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 283/538] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Sep 6 19:59:24 2022] GET /identity/v3/role_assignments?user.id=5776cbf4f6714b2083c3ebdd6d98cc87&scope.project.id=6aef43493bb1420a9e122313d8db7d5a&role.id=b950bf73c93e4f7ebed923ff14e39faa => generated 259 bytes in 551 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:24.717547 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038f04be-acfe-4fad-8c37-ebab53f2ff9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:24.719267 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038f04be-acfe-4fad-8c37-ebab53f2ff9a None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:24.720138 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038f04be-acfe-4fad-8c37-ebab53f2ff9a None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:24.721361 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038f04be-acfe-4fad-8c37-ebab53f2ff9a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:24.740288 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038f04be-acfe-4fad-8c37-ebab53f2ff9a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:24.743556 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038f04be-acfe-4fad-8c37-ebab53f2ff9a None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:24.778872 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-038f04be-acfe-4fad-8c37-ebab53f2ff9a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 19:59:24.778872 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:24.778872 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:24.778872 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:24.778872 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:24.778872 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:24.778872 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:24.778872 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:24.778872 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:24.778872 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:24.778872 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:24.778872 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:24.778872 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 19:59:24.778872 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 19:59:24.778872 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 19:59:24.780537 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 19:59:24.780537 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:24.780537 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:24.780537 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:24.780537 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:24.780537 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:24.780537 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:24.780537 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:24.780537 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:24.780537 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:24.780537 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:24.780537 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:24.780537 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:24.780537 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:24.780537 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:24.780537 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:24.780537 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:24.782287 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 19:59:24.782287 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 19:59:24.782287 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 19:59:24.782287 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 19:59:24.782287 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 19:59:24.782287 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 19:59:24.782287 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 19:59:24.782287 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 19:59:24.791183 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 256/539] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 6 19:59:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 461 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:59:24.803516 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff75a515-deee-4a7f-8bd5-675c003a7a49 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:25.073826 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff75a515-deee-4a7f-8bd5-675c003a7a49 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:25.106006 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:25.149299 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2623540-1be6-4cba-9c85-0cb5c3dc8f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:25.150376 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2623540-1be6-4cba-9c85-0cb5c3dc8f6e None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:25.150877 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2623540-1be6-4cba-9c85-0cb5c3dc8f6e None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:25.151369 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2623540-1be6-4cba-9c85-0cb5c3dc8f6e None admin] PATH_INFO: `/v3/projects` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:25.152713 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2623540-1be6-4cba-9c85-0cb5c3dc8f6e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:25.153378 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2623540-1be6-4cba-9c85-0cb5c3dc8f6e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:25.156092 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2623540-1be6-4cba-9c85-0cb5c3dc8f6e None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:25.192450 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 284/540] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Sep 6 19:59:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 390 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:25.202994 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038f04be-acfe-4fad-8c37-ebab53f2ff9a None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:25.371599 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d0eac6-34ee-43cd-8a61-de450184f027 None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:25.372416 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d0eac6-34ee-43cd-8a61-de450184f027 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:25.372561 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d0eac6-34ee-43cd-8a61-de450184f027 None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:25.375224 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 285/541] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 19:59:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 19:59:25.387907 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824994e7-b7b4-4a5e-b295-b7f727641153 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:25.388361 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824994e7-b7b4-4a5e-b295-b7f727641153 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:25.388832 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824994e7-b7b4-4a5e-b295-b7f727641153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:25.408397 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:25.435447 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038f04be-acfe-4fad-8c37-ebab53f2ff9a None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:25.443370 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-824994e7-b7b4-4a5e-b295-b7f727641153 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 19:59:25.445363 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-824994e7-b7b4-4a5e-b295-b7f727641153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 19:59:25.515039 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13435fb-62c8-4833-94b0-592962a6bf9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:25.516331 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13435fb-62c8-4833-94b0-592962a6bf9e None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:25.516830 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13435fb-62c8-4833-94b0-592962a6bf9e None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:25.517488 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13435fb-62c8-4833-94b0-592962a6bf9e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:25.519326 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13435fb-62c8-4833-94b0-592962a6bf9e None admin] RBAC: Adding query filter params (role.id=b950bf73c93e4f7ebed923ff14e39faa, scope.project.id=6aef43493bb1420a9e122313d8db7d5a, user.id=32882d38295046fea004a659a71bde58) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:25.519892 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13435fb-62c8-4833-94b0-592962a6bf9e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:25.527518 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13435fb-62c8-4833-94b0-592962a6bf9e None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:25.549929 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 257/542] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Sep 6 19:59:25 2022] GET /identity/v3/role_assignments?user.id=32882d38295046fea004a659a71bde58&scope.project.id=6aef43493bb1420a9e122313d8db7d5a&role.id=b950bf73c93e4f7ebed923ff14e39faa => generated 259 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:25.637737 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 286/543] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 19:59:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 19:59:26.008160 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13435fb-62c8-4833-94b0-592962a6bf9e None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:26.164431 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13435fb-62c8-4833-94b0-592962a6bf9e None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:26.215660 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecc3df7-8516-457f-96a9-24edb08bcf24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:26.216760 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecc3df7-8516-457f-96a9-24edb08bcf24 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:26.217250 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecc3df7-8516-457f-96a9-24edb08bcf24 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:26.217768 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecc3df7-8516-457f-96a9-24edb08bcf24 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:26.247374 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecc3df7-8516-457f-96a9-24edb08bcf24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 19:59:26.258656 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-eecc3df7-8516-457f-96a9-24edb08bcf24 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 19:59:26.258656 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:26.258656 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:26.258656 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:26.258656 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:26.258656 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:26.258656 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:26.258656 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:26.258656 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:26.258656 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:26.258656 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:26.258656 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:26.258656 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 19:59:26.258656 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 19:59:26.258656 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 19:59:26.258656 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 19:59:26.260382 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 19:59:26.260382 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 19:59:26.260382 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 19:59:26.260382 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 19:59:26.260382 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:26.260382 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:26.260382 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:26.260382 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:26.260382 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:26.260382 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:26.260382 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:26.260382 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:26.260382 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:26.260382 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:26.260382 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:26.260382 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:26.262478 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:26.262478 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:26.262478 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:26.262478 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:26.262478 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 19:59:26.262478 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 19:59:26.262478 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 19:59:26.262478 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 19:59:26.262478 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 19:59:26.262478 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 19:59:26.262478 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 19:59:26.262478 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 19:59:26.262478 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 258/544] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Sep 6 19:59:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:59:26.270905 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824994e7-b7b4-4a5e-b295-b7f727641153 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:26.458121 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824994e7-b7b4-4a5e-b295-b7f727641153 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:26.478251 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:26.529283 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ff50dd-f567-4244-92a0-056f18327661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:26.529993 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ff50dd-f567-4244-92a0-056f18327661 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:26.530334 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ff50dd-f567-4244-92a0-056f18327661 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:26.530610 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ff50dd-f567-4244-92a0-056f18327661 None admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:26.531345 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ff50dd-f567-4244-92a0-056f18327661 None admin] RBAC: Adding query filter params (name=service) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:26.531710 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ff50dd-f567-4244-92a0-056f18327661 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:26.534655 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ff50dd-f567-4244-92a0-056f18327661 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:26.547235 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 287/545] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Sep 6 19:59:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:26.548329 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8b5080-b145-445c-8008-5f0b0c9a9736 None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:26.549161 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8b5080-b145-445c-8008-5f0b0c9a9736 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:26.549415 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8b5080-b145-445c-8008-5f0b0c9a9736 None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:26.552726 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 259/546] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 19:59:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 19:59:26.559641 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ff50dd-f567-4244-92a0-056f18327661 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:26.571518 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc41f80-50de-4b5b-96d8-13f99e674101 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:26.571518 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc41f80-50de-4b5b-96d8-13f99e674101 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:26.571518 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc41f80-50de-4b5b-96d8-13f99e674101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:26.623160 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-9dc41f80-50de-4b5b-96d8-13f99e674101 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 19:59:26.624273 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-9dc41f80-50de-4b5b-96d8-13f99e674101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 19:59:26.638305 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:26.730501 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ff50dd-f567-4244-92a0-056f18327661 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:26.781256 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18bd70c-b80c-4a6b-8ff7-ff5b4397ec17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:26.782349 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18bd70c-b80c-4a6b-8ff7-ff5b4397ec17 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:26.782785 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18bd70c-b80c-4a6b-8ff7-ff5b4397ec17 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:26.783255 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18bd70c-b80c-4a6b-8ff7-ff5b4397ec17 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:26.805460 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18bd70c-b80c-4a6b-8ff7-ff5b4397ec17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:26.807378 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18bd70c-b80c-4a6b-8ff7-ff5b4397ec17 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:26.832690 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-e18bd70c-b80c-4a6b-8ff7-ff5b4397ec17 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 19:59:26.832690 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:26.832690 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:26.832690 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:26.832690 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:26.832690 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:26.832690 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:26.832690 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:26.832690 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:26.832690 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:26.832690 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:26.832690 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:26.832690 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 19:59:26.832690 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 19:59:26.832690 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 19:59:26.832690 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 19:59:26.833727 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:26.833727 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:26.833727 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:26.833727 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:26.833727 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:26.833727 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:26.833727 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:26.833727 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:26.833727 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:26.833727 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:26.833727 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:26.833727 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:26.833727 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:26.833727 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:26.833727 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:26.833727 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:26.833727 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 19:59:26.835283 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 19:59:26.835283 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 19:59:26.835283 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 19:59:26.838488 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 288/547] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 19:59:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:59:26.847815 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18bd70c-b80c-4a6b-8ff7-ff5b4397ec17 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:26.852856 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 260/548] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 19:59:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 19:59:26.898237 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:27.046489 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18bd70c-b80c-4a6b-8ff7-ff5b4397ec17 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:27.073181 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4332bfbf-cd31-4fd9-a8cf-b99c8fff91f1 None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:27.073964 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4332bfbf-cd31-4fd9-a8cf-b99c8fff91f1 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:27.074542 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4332bfbf-cd31-4fd9-a8cf-b99c8fff91f1 None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:27.077171 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 261/549] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 19:59:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 19:59:27.085234 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6092688-ff49-47f5-a95b-2285ab3f3747 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:27.085921 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6092688-ff49-47f5-a95b-2285ab3f3747 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:27.086462 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6092688-ff49-47f5-a95b-2285ab3f3747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:27.138440 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-d6092688-ff49-47f5-a95b-2285ab3f3747 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 19:59:27.140629 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-d6092688-ff49-47f5-a95b-2285ab3f3747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 19:59:27.162750 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2f18f3-fe7d-4b5f-9265-ec947864eb72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:27.162750 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2f18f3-fe7d-4b5f-9265-ec947864eb72 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:27.162750 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2f18f3-fe7d-4b5f-9265-ec947864eb72 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:27.163122 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2f18f3-fe7d-4b5f-9265-ec947864eb72 None admin] PATH_INFO: `/v3/domains` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:27.164802 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2f18f3-fe7d-4b5f-9265-ec947864eb72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:27.165308 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2f18f3-fe7d-4b5f-9265-ec947864eb72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:27.168100 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2f18f3-fe7d-4b5f-9265-ec947864eb72 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:27.205853 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 289/550] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 6 19:59:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:27.214863 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2f18f3-fe7d-4b5f-9265-ec947864eb72 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:27.324031 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 262/551] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 19:59:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 19:59:27.325112 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6092688-ff49-47f5-a95b-2285ab3f3747 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:27.438069 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:27.458129 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2f18f3-fe7d-4b5f-9265-ec947864eb72 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:27.532071 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6341a5-4ab1-4d24-8e5a-8cc7c7619491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:27.533396 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6341a5-4ab1-4d24-8e5a-8cc7c7619491 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:27.533932 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6341a5-4ab1-4d24-8e5a-8cc7c7619491 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:27.534943 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6341a5-4ab1-4d24-8e5a-8cc7c7619491 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:27.550676 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6341a5-4ab1-4d24-8e5a-8cc7c7619491 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:27.553859 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6341a5-4ab1-4d24-8e5a-8cc7c7619491 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:27.590520 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-bf6341a5-4ab1-4d24-8e5a-8cc7c7619491 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 06 19:59:27.590520 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:27.590520 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:27.590520 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:27.590520 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:27.590520 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:27.590520 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:27.590520 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:27.590520 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:27.590520 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:27.590520 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:27.590520 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:27.590520 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 19:59:27.590520 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 19:59:27.590520 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 19:59:27.590520 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 19:59:27.592679 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:27.592679 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:27.592679 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 19:59:27.592679 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 19:59:27.592679 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 19:59:27.592679 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 19:59:27.592679 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:27.592679 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:27.592679 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:27.592679 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:27.592679 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:27.592679 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:27.592679 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:27.592679 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:27.592679 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:27.592679 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:27.592679 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:27.592679 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:27.595041 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:27.595041 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:27.595041 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 19:59:27.595041 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 19:59:27.595041 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 19:59:27.595041 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 19:59:27.595041 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 19:59:27.595041 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 19:59:27.595041 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 06 19:59:27.595041 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 19:59:27.599540 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 290/552] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Sep 6 19:59:27 2022] GET /identity/v3/users/swift => generated 83 bytes in 386 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:59:27.605726 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6341a5-4ab1-4d24-8e5a-8cc7c7619491 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:27.635813 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:27.734725 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6092688-ff49-47f5-a95b-2285ab3f3747 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:27.806175 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3796dedc-c0d8-411d-a62f-c843cafb686b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:27.807443 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3796dedc-c0d8-411d-a62f-c843cafb686b None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:27.808042 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3796dedc-c0d8-411d-a62f-c843cafb686b None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:27.808645 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3796dedc-c0d8-411d-a62f-c843cafb686b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:27.830487 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3796dedc-c0d8-411d-a62f-c843cafb686b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 19:59:27.837585 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6341a5-4ab1-4d24-8e5a-8cc7c7619491 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:27.841694 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-3796dedc-c0d8-411d-a62f-c843cafb686b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 19:59:27.841694 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:27.841694 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:27.841694 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:27.841694 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:27.841694 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:27.841694 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:27.841694 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:27.841694 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:27.841694 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:27.841694 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:27.841694 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:27.841694 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 19:59:27.841694 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 19:59:27.841694 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 19:59:27.841694 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 19:59:27.843697 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 19:59:27.843697 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 19:59:27.843697 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 19:59:27.843697 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 19:59:27.843697 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:27.843697 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:27.843697 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:27.843697 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:27.843697 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:27.843697 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:27.843697 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:27.843697 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:27.843697 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:27.843697 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:27.843697 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:27.843697 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:27.845231 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:27.845231 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:27.845231 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:27.845231 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:27.845231 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 19:59:27.845231 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 19:59:27.845231 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 19:59:27.845231 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 19:59:27.845231 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 19:59:27.845231 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 19:59:27.845231 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 19:59:27.845231 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 19:59:27.847470 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 263/553] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Sep 6 19:59:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 522 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:59:27.856649 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3796dedc-c0d8-411d-a62f-c843cafb686b None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:27.912410 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51a004e-b74d-4cf5-83a5-efd70aef902c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:27.913906 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51a004e-b74d-4cf5-83a5-efd70aef902c None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:27.914460 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51a004e-b74d-4cf5-83a5-efd70aef902c None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:27.914974 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51a004e-b74d-4cf5-83a5-efd70aef902c None admin] PATH_INFO: `/v3/users` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:27.916656 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51a004e-b74d-4cf5-83a5-efd70aef902c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:27.917551 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51a004e-b74d-4cf5-83a5-efd70aef902c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:27.920961 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51a004e-b74d-4cf5-83a5-efd70aef902c None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:27.942337 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:27.957673 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 291/554] 10.222.0.20 () {56 vars in 1135 bytes} [Tue Sep 6 19:59:27 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:27.959736 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51a004e-b74d-4cf5-83a5-efd70aef902c None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:28.067683 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3796dedc-c0d8-411d-a62f-c843cafb686b None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:28.130188 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:28.139980 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dce3f0-8463-4e7f-93ab-31ccbf463e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:28.142030 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dce3f0-8463-4e7f-93ab-31ccbf463e72 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:28.142030 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dce3f0-8463-4e7f-93ab-31ccbf463e72 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:28.142560 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dce3f0-8463-4e7f-93ab-31ccbf463e72 None admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:28.144051 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dce3f0-8463-4e7f-93ab-31ccbf463e72 None admin] RBAC: Adding query filter params (name=service) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:28.144689 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dce3f0-8463-4e7f-93ab-31ccbf463e72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:28.147715 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dce3f0-8463-4e7f-93ab-31ccbf463e72 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:28.170089 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 264/555] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Sep 6 19:59:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:28.171965 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dce3f0-8463-4e7f-93ab-31ccbf463e72 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:28.187905 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51a004e-b74d-4cf5-83a5-efd70aef902c None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:28.262167 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9e8152-6a85-4d9c-9427-33420ea10f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:28.263580 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9e8152-6a85-4d9c-9427-33420ea10f8d None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:28.264311 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9e8152-6a85-4d9c-9427-33420ea10f8d None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:28.265104 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9e8152-6a85-4d9c-9427-33420ea10f8d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:28.285951 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9e8152-6a85-4d9c-9427-33420ea10f8d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 19:59:28.296358 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-cc9e8152-6a85-4d9c-9427-33420ea10f8d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 19:59:28.296358 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:28.296358 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:28.296358 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:28.296358 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:28.296358 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:28.296358 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:28.296358 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:28.296358 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:28.296358 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:28.296358 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:28.296358 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:28.296358 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 19:59:28.296358 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 19:59:28.296358 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 19:59:28.296358 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 19:59:28.298201 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 19:59:28.298201 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 19:59:28.298201 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 19:59:28.298201 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 19:59:28.298201 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:28.298201 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:28.298201 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:28.298201 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:28.298201 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:28.298201 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:28.298201 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:28.298201 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:28.298201 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:28.298201 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:28.298201 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:28.298201 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:28.299981 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:28.299981 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:28.299981 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:28.299981 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:28.299981 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 19:59:28.299981 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 19:59:28.299981 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 19:59:28.299981 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 19:59:28.299981 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 19:59:28.299981 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 19:59:28.299981 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 19:59:28.299981 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 19:59:28.303740 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 292/556] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Sep 6 19:59:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:59:28.305501 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9e8152-6a85-4d9c-9427-33420ea10f8d None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:28.378825 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dce3f0-8463-4e7f-93ab-31ccbf463e72 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:28.416909 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:28.436236 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90573632-7cf4-4f20-9f3d-cffbb76bcd9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:28.437174 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90573632-7cf4-4f20-9f3d-cffbb76bcd9f None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:28.437646 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90573632-7cf4-4f20-9f3d-cffbb76bcd9f None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:28.438101 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90573632-7cf4-4f20-9f3d-cffbb76bcd9f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:28.494995 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9e8152-6a85-4d9c-9427-33420ea10f8d None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:28.546625 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e0350b-13c8-4af0-a748-d79034a9ccec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:28.547651 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e0350b-13c8-4af0-a748-d79034a9ccec None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:28.548096 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e0350b-13c8-4af0-a748-d79034a9ccec None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:28.548517 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e0350b-13c8-4af0-a748-d79034a9ccec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:28.552147 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:28.564900 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e0350b-13c8-4af0-a748-d79034a9ccec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:28.566595 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e0350b-13c8-4af0-a748-d79034a9ccec None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:28.574753 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90573632-7cf4-4f20-9f3d-cffbb76bcd9f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:28.577474 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90573632-7cf4-4f20-9f3d-cffbb76bcd9f None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:28.586259 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-10e0350b-13c8-4af0-a748-d79034a9ccec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 19:59:28.586259 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:28.586259 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:28.586259 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:28.586259 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:28.586259 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:28.586259 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:28.586259 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:28.586259 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:28.586259 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:28.586259 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:28.586259 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:28.586259 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 19:59:28.586259 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 19:59:28.586259 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 19:59:28.586259 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 19:59:28.595165 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:28.595165 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:28.595165 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:28.595165 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:28.595165 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:28.595165 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:28.595165 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:28.595165 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:28.595165 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:28.595165 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:28.595165 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:28.595165 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:28.595165 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:28.595165 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:28.595165 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:28.595165 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:28.595165 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 19:59:28.596990 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 19:59:28.596990 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 19:59:28.596990 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 19:59:28.596990 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 293/557] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 19:59:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:59:28.596990 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e0350b-13c8-4af0-a748-d79034a9ccec None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:28.753778 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 265/558] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 6 19:59:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 576 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 19:59:28.753778 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bd47d2-04a4-4a52-8896-2e3ec295e630 None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:28.753778 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bd47d2-04a4-4a52-8896-2e3ec295e630 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:28.753778 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bd47d2-04a4-4a52-8896-2e3ec295e630 None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:28.753778 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 266/559] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 19:59:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 19:59:28.754980 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bd47d2-04a4-4a52-8896-2e3ec295e630 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:28.794190 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e0350b-13c8-4af0-a748-d79034a9ccec None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:28.809512 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:28.845201 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f19933c-29bc-4110-8da8-0cb5b1863082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:28.845978 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f19933c-29bc-4110-8da8-0cb5b1863082 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:28.846279 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f19933c-29bc-4110-8da8-0cb5b1863082 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:28.847149 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f19933c-29bc-4110-8da8-0cb5b1863082 None admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:28.847907 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f19933c-29bc-4110-8da8-0cb5b1863082 None admin] RBAC: Adding query filter params (name=service) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:28.848249 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f19933c-29bc-4110-8da8-0cb5b1863082 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:28.850011 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f19933c-29bc-4110-8da8-0cb5b1863082 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:28.860974 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 294/560] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Sep 6 19:59:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:28.862570 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f19933c-29bc-4110-8da8-0cb5b1863082 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:28.954248 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:28.968560 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bd47d2-04a4-4a52-8896-2e3ec295e630 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:29.010763 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f19933c-29bc-4110-8da8-0cb5b1863082 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:29.023011 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22cba2d-a495-476d-b14f-b13dc460c80b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:29.023778 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22cba2d-a495-476d-b14f-b13dc460c80b None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:29.024147 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22cba2d-a495-476d-b14f-b13dc460c80b None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:29.024517 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22cba2d-a495-476d-b14f-b13dc460c80b None admin] PATH_INFO: `/v3/domains` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:29.025339 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22cba2d-a495-476d-b14f-b13dc460c80b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:29.025895 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22cba2d-a495-476d-b14f-b13dc460c80b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:29.027576 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22cba2d-a495-476d-b14f-b13dc460c80b None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:29.047504 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 267/561] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 6 19:59:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:29.049565 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f984aee-7a90-4984-8e5d-e6f6ce6e2464 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:29.050147 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f984aee-7a90-4984-8e5d-e6f6ce6e2464 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:29.050604 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f984aee-7a90-4984-8e5d-e6f6ce6e2464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:29.067127 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d47530-961c-41f3-bd0c-cbfc52c6f5d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:29.067877 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d47530-961c-41f3-bd0c-cbfc52c6f5d8 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:29.068224 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d47530-961c-41f3-bd0c-cbfc52c6f5d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:29.068563 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d47530-961c-41f3-bd0c-cbfc52c6f5d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:29.075329 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-4f984aee-7a90-4984-8e5d-e6f6ce6e2464 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 19:59:29.076795 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-4f984aee-7a90-4984-8e5d-e6f6ce6e2464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 19:59:29.084225 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d47530-961c-41f3-bd0c-cbfc52c6f5d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:29.086049 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d47530-961c-41f3-bd0c-cbfc52c6f5d8 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:29.103595 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-51d47530-961c-41f3-bd0c-cbfc52c6f5d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 19:59:29.103595 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:29.103595 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:29.103595 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:29.103595 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:29.103595 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:29.103595 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:29.103595 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:29.103595 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:29.103595 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:29.103595 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:29.103595 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:29.103595 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 19:59:29.103595 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 19:59:29.103595 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 19:59:29.103595 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 19:59:29.105707 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:29.105707 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:29.105707 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:29.105707 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:29.105707 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:29.105707 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:29.105707 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:29.105707 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:29.105707 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:29.105707 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:29.105707 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:29.105707 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:29.105707 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:29.105707 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:29.105707 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:29.105707 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:29.105707 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 19:59:29.107683 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 19:59:29.107683 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 19:59:29.107683 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 19:59:29.107683 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 295/562] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 19:59:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:59:29.107683 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d47530-961c-41f3-bd0c-cbfc52c6f5d8 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:29.146289 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:29.186492 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 268/563] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 19:59:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 19:59:29.188925 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b150749-8eb0-4a1b-9360-74753f22cd02 None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:29.189435 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b150749-8eb0-4a1b-9360-74753f22cd02 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:29.189935 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b150749-8eb0-4a1b-9360-74753f22cd02 None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:29.192630 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 269/564] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 19:59:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 19:59:29.194300 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b150749-8eb0-4a1b-9360-74753f22cd02 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:29.266029 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d47530-961c-41f3-bd0c-cbfc52c6f5d8 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:29.324465 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a43b84-a11b-497e-8887-4ae35544ec09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:29.325079 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a43b84-a11b-497e-8887-4ae35544ec09 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:29.325360 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a43b84-a11b-497e-8887-4ae35544ec09 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:29.325675 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a43b84-a11b-497e-8887-4ae35544ec09 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:29.340001 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a43b84-a11b-497e-8887-4ae35544ec09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:29.341539 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a43b84-a11b-497e-8887-4ae35544ec09 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:29.359607 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-26a43b84-a11b-497e-8887-4ae35544ec09 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 19:59:29.359607 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:29.359607 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:29.359607 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:29.359607 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:29.359607 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:29.359607 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:29.359607 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:29.359607 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:29.359607 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:29.359607 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:29.359607 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:29.359607 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 19:59:29.359607 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 19:59:29.359607 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 19:59:29.359607 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 19:59:29.363152 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:29.363152 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:29.363152 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:29.363152 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:29.363152 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:29.363152 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:29.363152 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:29.363152 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:29.363152 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:29.363152 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:29.363152 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:29.363152 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:29.363152 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:29.363152 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:29.363152 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:29.363152 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:29.363152 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 19:59:29.365682 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 19:59:29.365682 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 19:59:29.365682 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 19:59:29.366620 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 296/565] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 19:59:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:59:29.367625 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a43b84-a11b-497e-8887-4ae35544ec09 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:29.396391 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:29.423027 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b150749-8eb0-4a1b-9360-74753f22cd02 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:29.472911 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9846793-b1e9-4977-8da5-1ad7932dc348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:29.473572 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9846793-b1e9-4977-8da5-1ad7932dc348 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:29.473886 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9846793-b1e9-4977-8da5-1ad7932dc348 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:29.474153 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9846793-b1e9-4977-8da5-1ad7932dc348 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:29.484966 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9846793-b1e9-4977-8da5-1ad7932dc348 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:29.486676 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9846793-b1e9-4977-8da5-1ad7932dc348 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:29.502219 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-c9846793-b1e9-4977-8da5-1ad7932dc348 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 06 19:59:29.502219 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:29.502219 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:29.502219 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:29.502219 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:29.502219 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:29.502219 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:29.502219 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:29.502219 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:29.502219 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:29.502219 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:29.502219 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:29.502219 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 19:59:29.502219 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 19:59:29.502219 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 19:59:29.502219 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 19:59:29.504686 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:29.504686 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:29.504686 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 19:59:29.504686 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 19:59:29.504686 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 19:59:29.504686 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 19:59:29.504686 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:29.504686 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:29.504686 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:29.504686 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:29.504686 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:29.504686 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:29.504686 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:29.504686 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:29.504686 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:29.504686 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:29.504686 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:29.504686 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:29.507360 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:29.507360 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:29.507360 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 19:59:29.507360 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 19:59:29.507360 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 19:59:29.507360 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 19:59:29.507360 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 19:59:29.507360 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 19:59:29.507360 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 06 19:59:29.507360 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 19:59:29.507360 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 270/566] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 6 19:59:29 2022] GET /identity/v3/users/glance => generated 84 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:59:29.507360 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20498375-c85b-4bc0-a80b-f5ebcd2d875e None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:29.507360 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20498375-c85b-4bc0-a80b-f5ebcd2d875e None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:29.510695 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20498375-c85b-4bc0-a80b-f5ebcd2d875e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:29.526959 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:29.528131 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-20498375-c85b-4bc0-a80b-f5ebcd2d875e None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 19:59:29.529499 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-20498375-c85b-4bc0-a80b-f5ebcd2d875e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 19:59:29.564797 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a43b84-a11b-497e-8887-4ae35544ec09 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:29.612584 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36472273-6051-46a3-ae1b-2caadbc085a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:29.613213 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36472273-6051-46a3-ae1b-2caadbc085a9 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:29.613532 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36472273-6051-46a3-ae1b-2caadbc085a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:29.613897 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36472273-6051-46a3-ae1b-2caadbc085a9 None admin] PATH_INFO: `/v3/domains` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:29.614753 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36472273-6051-46a3-ae1b-2caadbc085a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:29.615068 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36472273-6051-46a3-ae1b-2caadbc085a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:29.616704 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36472273-6051-46a3-ae1b-2caadbc085a9 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:29.627892 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 271/567] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 19:59:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 19:59:29.629717 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20498375-c85b-4bc0-a80b-f5ebcd2d875e None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:29.636489 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 297/568] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 6 19:59:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:29.636489 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36472273-6051-46a3-ae1b-2caadbc085a9 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:29.754230 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:29.796824 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20498375-c85b-4bc0-a80b-f5ebcd2d875e None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:29.845164 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36472273-6051-46a3-ae1b-2caadbc085a9 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:29.867417 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5f3c4a-c7ec-4eed-b7b5-871e6643f716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:29.868387 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5f3c4a-c7ec-4eed-b7b5-871e6643f716 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:29.868838 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5f3c4a-c7ec-4eed-b7b5-871e6643f716 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:29.869269 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5f3c4a-c7ec-4eed-b7b5-871e6643f716 None admin] PATH_INFO: `/v3/domains` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:29.870478 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5f3c4a-c7ec-4eed-b7b5-871e6643f716 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:29.870988 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5f3c4a-c7ec-4eed-b7b5-871e6643f716 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:29.873717 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5f3c4a-c7ec-4eed-b7b5-871e6643f716 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:29.890278 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 272/569] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 6 19:59:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:29.892288 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5f3c4a-c7ec-4eed-b7b5-871e6643f716 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:29.916925 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2329fc-7f5c-4ebb-bc95-42367873bf5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:29.920585 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2329fc-7f5c-4ebb-bc95-42367873bf5f None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:29.920585 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2329fc-7f5c-4ebb-bc95-42367873bf5f None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:29.920585 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2329fc-7f5c-4ebb-bc95-42367873bf5f None admin] PATH_INFO: `/v3/users` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:29.920585 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2329fc-7f5c-4ebb-bc95-42367873bf5f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:29.921396 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2329fc-7f5c-4ebb-bc95-42367873bf5f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:29.923393 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2329fc-7f5c-4ebb-bc95-42367873bf5f None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:29.930466 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:29.962086 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 298/570] 10.222.0.20 () {56 vars in 1137 bytes} [Tue Sep 6 19:59:29 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:29.963484 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2329fc-7f5c-4ebb-bc95-42367873bf5f None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:30.062151 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5f3c4a-c7ec-4eed-b7b5-871e6643f716 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:30.109785 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014288bc-73c9-47df-a745-bb7ba5609e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:30.110777 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014288bc-73c9-47df-a745-bb7ba5609e42 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:30.111149 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014288bc-73c9-47df-a745-bb7ba5609e42 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:30.111609 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014288bc-73c9-47df-a745-bb7ba5609e42 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:30.115073 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:30.132153 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014288bc-73c9-47df-a745-bb7ba5609e42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 19:59:30.139401 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-014288bc-73c9-47df-a745-bb7ba5609e42 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 19:59:30.139401 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:30.139401 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:30.139401 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:30.139401 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:30.139401 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:30.139401 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:30.139401 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:30.139401 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:30.139401 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:30.139401 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:30.139401 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:30.139401 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 19:59:30.139401 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 19:59:30.139401 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 19:59:30.139401 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 19:59:30.141902 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 19:59:30.141902 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 19:59:30.141902 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 19:59:30.141902 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 19:59:30.141902 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:30.141902 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:30.141902 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:30.141902 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:30.141902 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:30.141902 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:30.141902 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:30.141902 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:30.141902 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:30.141902 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:30.141902 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:30.141902 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:30.144810 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:30.144810 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:30.144810 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:30.144810 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:30.144810 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 19:59:30.144810 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 19:59:30.144810 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 19:59:30.144810 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 19:59:30.144810 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 19:59:30.144810 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 19:59:30.144810 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 19:59:30.144810 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 19:59:30.144810 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 273/571] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Sep 6 19:59:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:59:30.144810 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014288bc-73c9-47df-a745-bb7ba5609e42 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:30.147510 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2329fc-7f5c-4ebb-bc95-42367873bf5f None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:30.210522 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf540ae-6ae2-473e-85b6-875b1893fc48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:30.211437 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf540ae-6ae2-473e-85b6-875b1893fc48 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:30.211765 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf540ae-6ae2-473e-85b6-875b1893fc48 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:30.212198 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf540ae-6ae2-473e-85b6-875b1893fc48 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:30.223938 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf540ae-6ae2-473e-85b6-875b1893fc48 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:30.226460 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf540ae-6ae2-473e-85b6-875b1893fc48 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:30.246709 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-ddf540ae-6ae2-473e-85b6-875b1893fc48 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 19:59:30.246709 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:30.246709 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:30.246709 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:30.246709 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:30.246709 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:30.246709 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:30.246709 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:30.246709 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:30.246709 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:30.246709 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:30.246709 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:30.246709 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 19:59:30.246709 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 19:59:30.246709 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 19:59:30.248445 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 19:59:30.248445 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:30.248445 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:30.248445 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:30.248445 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:30.248445 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:30.248445 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:30.248445 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:30.248445 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:30.248445 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:30.248445 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:30.248445 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:30.248445 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:30.248445 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:30.248445 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:30.248445 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:30.248445 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:30.250963 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 19:59:30.250963 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 19:59:30.250963 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 19:59:30.250963 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 19:59:30.250963 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 19:59:30.250963 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 19:59:30.250963 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 19:59:30.250963 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 19:59:30.250963 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 299/572] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 6 19:59:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:59:30.250963 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf540ae-6ae2-473e-85b6-875b1893fc48 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:30.309107 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014288bc-73c9-47df-a745-bb7ba5609e42 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:30.329419 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:30.355017 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c5ff46-eab8-49e8-a5d0-332707473f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:30.355663 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c5ff46-eab8-49e8-a5d0-332707473f5d None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:30.355961 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c5ff46-eab8-49e8-a5d0-332707473f5d None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:30.356222 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c5ff46-eab8-49e8-a5d0-332707473f5d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:30.365038 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c5ff46-eab8-49e8-a5d0-332707473f5d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:30.366565 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c5ff46-eab8-49e8-a5d0-332707473f5d None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:30.380358 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-05c5ff46-eab8-49e8-a5d0-332707473f5d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 06 19:59:30.380358 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:30.380358 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:30.380358 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:30.380358 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:30.380358 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:30.380358 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:30.380358 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:30.380358 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:30.380358 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:30.380358 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:30.380358 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:30.380358 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 19:59:30.380358 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 19:59:30.380358 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 19:59:30.380358 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 19:59:30.389946 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:30.389946 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:30.389946 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 19:59:30.389946 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 19:59:30.389946 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 19:59:30.389946 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 19:59:30.389946 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:30.389946 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:30.389946 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:30.389946 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:30.389946 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:30.389946 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:30.389946 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:30.389946 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:30.389946 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:30.389946 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:30.389946 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:30.389946 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:30.392088 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:30.392088 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:30.392088 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 19:59:30.392088 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 19:59:30.392088 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 19:59:30.392088 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 19:59:30.392088 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 19:59:30.392088 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 19:59:30.392088 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 06 19:59:30.392088 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 19:59:30.392088 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 274/573] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 6 19:59:30 2022] GET /identity/v3/users/cinder => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:59:30.392088 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c5ff46-eab8-49e8-a5d0-332707473f5d None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:30.399202 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf540ae-6ae2-473e-85b6-875b1893fc48 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:30.448014 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28847b6e-5134-4206-a834-272c564fc3d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:30.448670 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28847b6e-5134-4206-a834-272c564fc3d5 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:30.448952 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28847b6e-5134-4206-a834-272c564fc3d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:30.449231 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28847b6e-5134-4206-a834-272c564fc3d5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:30.463949 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28847b6e-5134-4206-a834-272c564fc3d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 19:59:30.469132 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:30.470262 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-28847b6e-5134-4206-a834-272c564fc3d5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 19:59:30.470262 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:30.470262 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:30.470262 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:30.470262 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:30.470262 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:30.470262 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:30.470262 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:30.470262 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:30.470262 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:30.470262 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:30.470262 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:30.470262 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 19:59:30.470262 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 19:59:30.470262 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 19:59:30.470262 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 19:59:30.472568 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 19:59:30.472568 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 19:59:30.472568 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 19:59:30.472568 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 19:59:30.472568 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:30.472568 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:30.472568 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:30.472568 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:30.472568 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:30.472568 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:30.472568 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:30.472568 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:30.472568 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:30.472568 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:30.472568 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:30.472568 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:30.474323 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:30.474323 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:30.474323 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:30.474323 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:30.474323 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 19:59:30.474323 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 19:59:30.474323 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 19:59:30.474323 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 19:59:30.474323 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 19:59:30.474323 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 19:59:30.474323 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 19:59:30.474323 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 19:59:30.474323 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 300/574] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Sep 6 19:59:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:59:30.474323 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28847b6e-5134-4206-a834-272c564fc3d5 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:30.539233 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c5ff46-eab8-49e8-a5d0-332707473f5d None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:30.582663 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdb7a0d-80a6-4cec-b5a4-19f7e579ded0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:30.583293 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb7a0d-80a6-4cec-b5a4-19f7e579ded0 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:30.583572 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb7a0d-80a6-4cec-b5a4-19f7e579ded0 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:30.583848 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb7a0d-80a6-4cec-b5a4-19f7e579ded0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:30.596474 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdb7a0d-80a6-4cec-b5a4-19f7e579ded0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:30.598116 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdb7a0d-80a6-4cec-b5a4-19f7e579ded0 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:30.614573 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-fbdb7a0d-80a6-4cec-b5a4-19f7e579ded0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 19:59:30.614573 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:30.614573 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:30.614573 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:30.614573 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:30.614573 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:30.614573 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:30.614573 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:30.614573 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:30.614573 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:30.614573 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:30.614573 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:30.614573 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 19:59:30.614573 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 19:59:30.614573 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 19:59:30.614573 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 19:59:30.616429 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:30.616429 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:30.616429 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:30.616429 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:30.616429 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:30.616429 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:30.616429 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:30.616429 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:30.616429 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:30.616429 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:30.616429 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:30.616429 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:30.616429 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:30.616429 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:30.616429 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:30.616429 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:30.616429 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 19:59:30.617412 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 19:59:30.617412 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 19:59:30.617412 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 19:59:30.620459 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 275/575] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 19:59:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:59:30.621442 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdb7a0d-80a6-4cec-b5a4-19f7e579ded0 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:30.624929 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28847b6e-5134-4206-a834-272c564fc3d5 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:30.647273 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:30.678979 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1854a7e4-04de-42a2-94a6-3ecf63996b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:30.679594 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1854a7e4-04de-42a2-94a6-3ecf63996b48 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:30.679880 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1854a7e4-04de-42a2-94a6-3ecf63996b48 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:30.680159 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1854a7e4-04de-42a2-94a6-3ecf63996b48 None admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:30.680845 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1854a7e4-04de-42a2-94a6-3ecf63996b48 None admin] RBAC: Adding query filter params (name=service) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:30.681153 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1854a7e4-04de-42a2-94a6-3ecf63996b48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:30.682596 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1854a7e4-04de-42a2-94a6-3ecf63996b48 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:30.692442 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 301/576] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Sep 6 19:59:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:30.693419 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1854a7e4-04de-42a2-94a6-3ecf63996b48 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:30.767535 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdb7a0d-80a6-4cec-b5a4-19f7e579ded0 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:30.777999 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:30.810695 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ccdf88-7840-41a3-a2de-6200600c0145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:30.811329 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ccdf88-7840-41a3-a2de-6200600c0145 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:30.811614 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ccdf88-7840-41a3-a2de-6200600c0145 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:30.811893 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ccdf88-7840-41a3-a2de-6200600c0145 None admin] PATH_INFO: `/v3/projects` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:30.812623 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ccdf88-7840-41a3-a2de-6200600c0145 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:30.812936 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ccdf88-7840-41a3-a2de-6200600c0145 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:30.814498 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ccdf88-7840-41a3-a2de-6200600c0145 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:30.841677 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 276/577] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Sep 6 19:59:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:30.841677 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ccdf88-7840-41a3-a2de-6200600c0145 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:30.853563 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1854a7e4-04de-42a2-94a6-3ecf63996b48 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:30.905768 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d806832c-d14c-4115-9e8b-fa512cc1be5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:30.906558 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d806832c-d14c-4115-9e8b-fa512cc1be5b None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:30.906997 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d806832c-d14c-4115-9e8b-fa512cc1be5b None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:30.907481 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d806832c-d14c-4115-9e8b-fa512cc1be5b None admin] PATH_INFO: `/v3/users` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:30.908879 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d806832c-d14c-4115-9e8b-fa512cc1be5b None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:30.909538 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d806832c-d14c-4115-9e8b-fa512cc1be5b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:30.912361 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d806832c-d14c-4115-9e8b-fa512cc1be5b None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:30.943898 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 302/578] 10.222.0.20 () {56 vars in 1137 bytes} [Tue Sep 6 19:59:30 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:30.945006 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d806832c-d14c-4115-9e8b-fa512cc1be5b None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:30.991247 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:30.992689 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ccdf88-7840-41a3-a2de-6200600c0145 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:31.049639 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94378c05-e38a-44fe-9163-b9134243a8a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:31.050360 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94378c05-e38a-44fe-9163-b9134243a8a0 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:31.050641 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94378c05-e38a-44fe-9163-b9134243a8a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:31.051097 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94378c05-e38a-44fe-9163-b9134243a8a0 None admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:31.051840 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94378c05-e38a-44fe-9163-b9134243a8a0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:31.052155 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94378c05-e38a-44fe-9163-b9134243a8a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:31.053601 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94378c05-e38a-44fe-9163-b9134243a8a0 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:31.063151 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 277/579] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Sep 6 19:59:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:31.064187 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94378c05-e38a-44fe-9163-b9134243a8a0 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:31.117195 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:31.121699 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d806832c-d14c-4115-9e8b-fa512cc1be5b None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:31.176792 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dfa9ac-1688-4adb-b10e-a397c4947249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:31.177536 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dfa9ac-1688-4adb-b10e-a397c4947249 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:31.177889 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dfa9ac-1688-4adb-b10e-a397c4947249 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:31.178236 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dfa9ac-1688-4adb-b10e-a397c4947249 None admin] PATH_INFO: `/v3/domains` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:31.179133 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dfa9ac-1688-4adb-b10e-a397c4947249 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:31.179532 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dfa9ac-1688-4adb-b10e-a397c4947249 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:31.181403 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dfa9ac-1688-4adb-b10e-a397c4947249 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:31.197510 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 303/580] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 6 19:59:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:31.199356 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dfa9ac-1688-4adb-b10e-a397c4947249 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:31.225064 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94378c05-e38a-44fe-9163-b9134243a8a0 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:31.268069 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7381775b-913d-4db0-b20e-f586668f64cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:31.268970 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7381775b-913d-4db0-b20e-f586668f64cc None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:31.269389 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7381775b-913d-4db0-b20e-f586668f64cc None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:31.269914 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7381775b-913d-4db0-b20e-f586668f64cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:31.289317 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7381775b-913d-4db0-b20e-f586668f64cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:31.291902 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7381775b-913d-4db0-b20e-f586668f64cc None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:31.313306 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-7381775b-913d-4db0-b20e-f586668f64cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 19:59:31.313306 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:31.313306 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:31.313306 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:31.313306 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:31.313306 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:31.313306 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:31.313306 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:31.313306 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:31.313306 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:31.313306 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:31.313306 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:31.313306 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 19:59:31.313306 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 19:59:31.313306 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 19:59:31.313306 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 19:59:31.315434 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:31.315434 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:31.315434 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:31.315434 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:31.315434 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:31.315434 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:31.315434 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:31.315434 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:31.315434 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:31.315434 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:31.315434 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:31.315434 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:31.315434 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:31.315434 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:31.315434 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:31.315434 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:31.315434 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 19:59:31.317589 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 19:59:31.317589 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 19:59:31.317589 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 19:59:31.317589 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 278/581] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 19:59:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:59:31.317589 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7381775b-913d-4db0-b20e-f586668f64cc None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:31.329156 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:31.341331 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dfa9ac-1688-4adb-b10e-a397c4947249 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:31.392266 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49328851-5a53-48f5-af1c-01e7b6f95e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:31.392808 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49328851-5a53-48f5-af1c-01e7b6f95e3c None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:31.393128 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49328851-5a53-48f5-af1c-01e7b6f95e3c None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:31.393365 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49328851-5a53-48f5-af1c-01e7b6f95e3c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:31.394652 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49328851-5a53-48f5-af1c-01e7b6f95e3c None admin] RBAC: Adding query filter params (role.id=b950bf73c93e4f7ebed923ff14e39faa, scope.project.id=6aef43493bb1420a9e122313d8db7d5a, user.id=2b72516236da47e292d78de201b0552b) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:31.394994 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49328851-5a53-48f5-af1c-01e7b6f95e3c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:31.396464 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49328851-5a53-48f5-af1c-01e7b6f95e3c None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:31.404008 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 304/582] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Sep 6 19:59:31 2022] GET /identity/v3/role_assignments?user.id=2b72516236da47e292d78de201b0552b&scope.project.id=6aef43493bb1420a9e122313d8db7d5a&role.id=b950bf73c93e4f7ebed923ff14e39faa => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:31.405273 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49328851-5a53-48f5-af1c-01e7b6f95e3c None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:31.462551 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:31.495937 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7381775b-913d-4db0-b20e-f586668f64cc None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:31.553206 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49328851-5a53-48f5-af1c-01e7b6f95e3c None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:31.554565 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e77f29-6a3b-4843-917f-334c17bc3f11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:31.561343 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e77f29-6a3b-4843-917f-334c17bc3f11 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:31.562095 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e77f29-6a3b-4843-917f-334c17bc3f11 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:31.562550 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e77f29-6a3b-4843-917f-334c17bc3f11 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:31.583351 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e77f29-6a3b-4843-917f-334c17bc3f11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:31.584977 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e77f29-6a3b-4843-917f-334c17bc3f11 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:31.606112 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-93e77f29-6a3b-4843-917f-334c17bc3f11 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 19:59:31.606112 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:31.606112 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:31.606112 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:31.606112 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:31.606112 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:31.606112 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:31.606112 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:31.606112 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:31.606112 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:31.606112 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:31.606112 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:31.606112 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 19:59:31.606112 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 19:59:31.606112 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 19:59:31.606112 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 19:59:31.607986 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:31.607986 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:31.607986 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:31.607986 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:31.607986 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:31.607986 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:31.607986 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:31.607986 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:31.607986 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:31.607986 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:31.607986 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:31.607986 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:31.607986 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:31.607986 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:31.607986 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:31.607986 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:31.607986 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 19:59:31.609771 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 19:59:31.609771 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 19:59:31.609771 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 19:59:31.609771 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 279/583] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 19:59:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:59:31.609771 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e77f29-6a3b-4843-917f-334c17bc3f11 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:31.618157 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09f5cc9-76ab-409d-8f2b-e759664fc1d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:31.619186 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09f5cc9-76ab-409d-8f2b-e759664fc1d5 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:31.619652 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09f5cc9-76ab-409d-8f2b-e759664fc1d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:31.620106 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09f5cc9-76ab-409d-8f2b-e759664fc1d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:31.636119 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09f5cc9-76ab-409d-8f2b-e759664fc1d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:31.638980 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09f5cc9-76ab-409d-8f2b-e759664fc1d5 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:31.662113 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-b09f5cc9-76ab-409d-8f2b-e759664fc1d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 19:59:31.662113 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:31.662113 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:31.662113 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:31.662113 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:31.662113 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:31.662113 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:31.662113 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:31.662113 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:31.662113 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:31.662113 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:31.662113 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:31.662113 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 19:59:31.662113 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 19:59:31.662113 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 19:59:31.662113 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 19:59:31.663987 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:31.663987 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:31.663987 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:31.663987 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:31.663987 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:31.663987 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:31.663987 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:31.663987 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:31.663987 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:31.663987 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:31.663987 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:31.663987 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:31.663987 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:31.663987 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:31.663987 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:31.663987 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:31.663987 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 19:59:31.665846 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 19:59:31.665846 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 19:59:31.665846 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 19:59:31.665846 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 305/584] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 19:59:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:59:31.665846 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09f5cc9-76ab-409d-8f2b-e759664fc1d5 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:31.690499 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:31.826241 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e77f29-6a3b-4843-917f-334c17bc3f11 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:31.828573 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09f5cc9-76ab-409d-8f2b-e759664fc1d5 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:31.876986 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7ac254-e99d-4399-8336-4b6f14f08a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:31.877785 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7ac254-e99d-4399-8336-4b6f14f08a66 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:31.878913 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7ac254-e99d-4399-8336-4b6f14f08a66 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:31.879305 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7ac254-e99d-4399-8336-4b6f14f08a66 None admin] PATH_INFO: `/v3/domains` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:31.880087 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7ac254-e99d-4399-8336-4b6f14f08a66 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:31.880687 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7ac254-e99d-4399-8336-4b6f14f08a66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:31.882560 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7ac254-e99d-4399-8336-4b6f14f08a66 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:31.886898 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:31.896024 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94620de4-15e5-4e24-8c18-06bcc6ddbe83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:31.897285 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94620de4-15e5-4e24-8c18-06bcc6ddbe83 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:31.897900 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94620de4-15e5-4e24-8c18-06bcc6ddbe83 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:31.898403 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94620de4-15e5-4e24-8c18-06bcc6ddbe83 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:31.902274 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 306/585] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 6 19:59:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:31.904294 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7ac254-e99d-4399-8336-4b6f14f08a66 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:31.918802 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94620de4-15e5-4e24-8c18-06bcc6ddbe83 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:31.921842 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94620de4-15e5-4e24-8c18-06bcc6ddbe83 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:31.938623 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-94620de4-15e5-4e24-8c18-06bcc6ddbe83 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 19:59:31.938623 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:31.938623 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:31.938623 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:31.938623 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:31.938623 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:31.938623 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:31.938623 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:31.938623 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:31.938623 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:31.938623 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:31.938623 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:31.938623 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 19:59:31.938623 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 19:59:31.938623 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 19:59:31.939650 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 19:59:31.939650 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:31.939650 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:31.939650 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:31.939650 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:31.939650 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:31.939650 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:31.939650 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:31.939650 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:31.939650 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:31.939650 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:31.939650 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:31.939650 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:31.939650 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:31.939650 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:31.939650 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:31.939650 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:31.940542 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 19:59:31.940542 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 19:59:31.940542 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 19:59:31.940542 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 19:59:31.940542 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 19:59:31.940542 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 19:59:31.940542 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 19:59:31.940542 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 19:59:31.950194 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 280/586] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 6 19:59:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:59:31.950194 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94620de4-15e5-4e24-8c18-06bcc6ddbe83 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:32.044724 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:32.115918 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7ac254-e99d-4399-8336-4b6f14f08a66 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:32.134198 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94620de4-15e5-4e24-8c18-06bcc6ddbe83 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:32.177937 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f3d4aa-6439-4aa4-9bfe-a09c55c207d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:32.178481 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f3d4aa-6439-4aa4-9bfe-a09c55c207d4 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:32.178883 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f3d4aa-6439-4aa4-9bfe-a09c55c207d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:32.179138 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f3d4aa-6439-4aa4-9bfe-a09c55c207d4 None admin] PATH_INFO: `/v3/domains` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:32.179832 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f3d4aa-6439-4aa4-9bfe-a09c55c207d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:32.180189 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f3d4aa-6439-4aa4-9bfe-a09c55c207d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:32.181691 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f3d4aa-6439-4aa4-9bfe-a09c55c207d4 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:32.199914 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a03c08-53fe-4f62-8c1d-a5a4edbbc35d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:32.200604 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a03c08-53fe-4f62-8c1d-a5a4edbbc35d None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:32.200883 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a03c08-53fe-4f62-8c1d-a5a4edbbc35d None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:32.201160 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a03c08-53fe-4f62-8c1d-a5a4edbbc35d None admin] PATH_INFO: `/v3/domains` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:32.201857 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a03c08-53fe-4f62-8c1d-a5a4edbbc35d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:32.202159 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a03c08-53fe-4f62-8c1d-a5a4edbbc35d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:32.210090 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 307/587] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 6 19:59:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:32.210352 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a03c08-53fe-4f62-8c1d-a5a4edbbc35d None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:32.210481 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f3d4aa-6439-4aa4-9bfe-a09c55c207d4 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:32.225099 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 281/588] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 6 19:59:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:32.231065 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a03c08-53fe-4f62-8c1d-a5a4edbbc35d None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:32.303374 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:32.377645 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a03c08-53fe-4f62-8c1d-a5a4edbbc35d None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:32.421600 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f3d4aa-6439-4aa4-9bfe-a09c55c207d4 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:32.446214 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3419e88d-8b5e-4c66-9ed4-d5ea861ae1eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:32.447254 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3419e88d-8b5e-4c66-9ed4-d5ea861ae1eb None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:32.447800 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3419e88d-8b5e-4c66-9ed4-d5ea861ae1eb None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:32.448279 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3419e88d-8b5e-4c66-9ed4-d5ea861ae1eb None admin] PATH_INFO: `/v3/projects` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:32.449643 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3419e88d-8b5e-4c66-9ed4-d5ea861ae1eb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:32.450174 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3419e88d-8b5e-4c66-9ed4-d5ea861ae1eb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:32.452910 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3419e88d-8b5e-4c66-9ed4-d5ea861ae1eb None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:32.468881 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:32.477121 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 282/589] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Sep 6 19:59:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:32.478735 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3419e88d-8b5e-4c66-9ed4-d5ea861ae1eb None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:32.480411 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb4f1de-c262-4a76-8e53-8e8426f9014c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:32.481527 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb4f1de-c262-4a76-8e53-8e8426f9014c None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:32.482048 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb4f1de-c262-4a76-8e53-8e8426f9014c None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:32.482517 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb4f1de-c262-4a76-8e53-8e8426f9014c None admin] PATH_INFO: `/v3/users/nova` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:32.498006 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb4f1de-c262-4a76-8e53-8e8426f9014c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:32.500965 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb4f1de-c262-4a76-8e53-8e8426f9014c None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:32.531315 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-abb4f1de-c262-4a76-8e53-8e8426f9014c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 06 19:59:32.531315 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:32.531315 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:32.531315 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:32.531315 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:32.531315 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:32.531315 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:32.531315 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:32.531315 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:32.531315 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:32.531315 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:32.531315 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:32.531315 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 19:59:32.531315 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 19:59:32.531315 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 19:59:32.531315 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 19:59:32.533203 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:32.533203 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:32.533203 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 19:59:32.533203 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 19:59:32.533203 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 19:59:32.533203 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 19:59:32.533203 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:32.533203 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:32.533203 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:32.533203 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:32.533203 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:32.533203 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:32.533203 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:32.533203 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:32.533203 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:32.533203 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:32.533203 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:32.533203 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:32.543409 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:32.543409 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:32.543409 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 19:59:32.543409 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 19:59:32.543409 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 19:59:32.543409 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 19:59:32.543409 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 19:59:32.543409 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 19:59:32.543409 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 06 19:59:32.543409 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 19:59:32.543409 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 308/590] 10.222.0.20 () {56 vars in 1093 bytes} [Tue Sep 6 19:59:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:59:32.543409 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb4f1de-c262-4a76-8e53-8e8426f9014c None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:32.622009 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3419e88d-8b5e-4c66-9ed4-d5ea861ae1eb None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:32.651532 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:32.686427 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685c00ff-0135-427c-b125-44bb9f5ab320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:32.687391 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685c00ff-0135-427c-b125-44bb9f5ab320 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:32.687877 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685c00ff-0135-427c-b125-44bb9f5ab320 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:32.688334 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685c00ff-0135-427c-b125-44bb9f5ab320 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:32.698260 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685c00ff-0135-427c-b125-44bb9f5ab320 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:32.699818 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685c00ff-0135-427c-b125-44bb9f5ab320 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:32.715159 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-685c00ff-0135-427c-b125-44bb9f5ab320 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 19:59:32.715159 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:32.715159 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:32.715159 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:32.715159 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:32.715159 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:32.715159 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:32.715159 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:32.715159 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:32.715159 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:32.715159 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:32.715159 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:32.715159 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 19:59:32.715159 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 19:59:32.715159 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 19:59:32.716937 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 19:59:32.716937 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:32.716937 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:32.716937 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:32.716937 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:32.716937 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:32.716937 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:32.716937 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:32.716937 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:32.716937 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:32.716937 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:32.716937 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:32.716937 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:32.716937 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:32.716937 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:32.716937 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:32.716937 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:32.718711 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 19:59:32.718711 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 19:59:32.718711 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 19:59:32.718711 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 19:59:32.718711 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 19:59:32.718711 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 19:59:32.718711 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 19:59:32.718711 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 19:59:32.718711 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 283/591] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 6 19:59:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:59:32.718711 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685c00ff-0135-427c-b125-44bb9f5ab320 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:32.746515 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb4f1de-c262-4a76-8e53-8e8426f9014c None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:32.808418 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3162c0e6-8973-4972-a8ec-72a1bde36cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:32.809421 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3162c0e6-8973-4972-a8ec-72a1bde36cb4 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:32.809937 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3162c0e6-8973-4972-a8ec-72a1bde36cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:32.810409 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3162c0e6-8973-4972-a8ec-72a1bde36cb4 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:32.822330 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3162c0e6-8973-4972-a8ec-72a1bde36cb4 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:32.824427 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3162c0e6-8973-4972-a8ec-72a1bde36cb4 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:32.847390 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-3162c0e6-8973-4972-a8ec-72a1bde36cb4 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 06 19:59:32.847390 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:32.847390 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:32.847390 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:32.847390 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:32.847390 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:32.847390 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:32.847390 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:32.847390 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:32.847390 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:32.847390 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:32.847390 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:32.847390 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 19:59:32.847390 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 19:59:32.847390 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 19:59:32.847390 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 19:59:32.849235 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:32.849235 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:32.849235 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 19:59:32.849235 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 19:59:32.849235 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 19:59:32.849235 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 19:59:32.849235 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:32.849235 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:32.849235 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:32.849235 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:32.849235 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:32.849235 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:32.849235 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:32.849235 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:32.849235 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:32.849235 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:32.849235 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:32.849235 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:32.851049 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:32.851049 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:32.851049 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 19:59:32.851049 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 19:59:32.851049 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 19:59:32.851049 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 19:59:32.851049 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 19:59:32.851049 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 19:59:32.851049 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 06 19:59:32.851049 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 19:59:32.851049 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 309/592] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Sep 6 19:59:32 2022] GET /identity/v3/users/neutron => generated 85 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:59:32.851049 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3162c0e6-8973-4972-a8ec-72a1bde36cb4 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:32.885663 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685c00ff-0135-427c-b125-44bb9f5ab320 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:32.949286 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:32.963509 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b08306-8f16-4909-92a6-54e490245d4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:32.964857 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b08306-8f16-4909-92a6-54e490245d4b None admin] REQUEST_METHOD: `PUT` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:32.965943 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b08306-8f16-4909-92a6-54e490245d4b None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:32.966431 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b08306-8f16-4909-92a6-54e490245d4b None admin] PATH_INFO: `/v3/projects/6aef43493bb1420a9e122313d8db7d5a/users/07af18f35ffc42a295278d5349e06381/roles/b950bf73c93e4f7ebed923ff14e39faa` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:32.988464 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b08306-8f16-4909-92a6-54e490245d4b None admin] RBAC: Authorizing `identity:create_grant(project_id=6aef43493bb1420a9e122313d8db7d5a, user_id=07af18f35ffc42a295278d5349e06381, role_id=b950bf73c93e4f7ebed923ff14e39faa)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:32.991014 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b08306-8f16-4909-92a6-54e490245d4b None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:33.013667 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 284/593] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Sep 6 19:59:32 2022] PUT /identity/v3/projects/6aef43493bb1420a9e122313d8db7d5a/users/07af18f35ffc42a295278d5349e06381/roles/b950bf73c93e4f7ebed923ff14e39faa => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 19:59:33.013667 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b08306-8f16-4909-92a6-54e490245d4b None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:33.062160 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:33.071207 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3162c0e6-8973-4972-a8ec-72a1bde36cb4 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:33.123044 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b498a3b-8002-42e0-8b16-471c5c5372dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:33.123849 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b498a3b-8002-42e0-8b16-471c5c5372dd None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:33.124221 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b498a3b-8002-42e0-8b16-471c5c5372dd None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:33.124641 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b498a3b-8002-42e0-8b16-471c5c5372dd None admin] PATH_INFO: `/v3/users` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:33.125526 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b498a3b-8002-42e0-8b16-471c5c5372dd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:33.125971 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b498a3b-8002-42e0-8b16-471c5c5372dd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:33.127661 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b498a3b-8002-42e0-8b16-471c5c5372dd None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:33.161839 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 310/594] 10.222.0.20 () {56 vars in 1133 bytes} [Tue Sep 6 19:59:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:33.161839 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b498a3b-8002-42e0-8b16-471c5c5372dd None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:33.235753 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b08306-8f16-4909-92a6-54e490245d4b None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:33.288668 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc3bb70-8ec7-43cc-b84a-867c236f9c2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:33.289726 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc3bb70-8ec7-43cc-b84a-867c236f9c2a None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:33.290295 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc3bb70-8ec7-43cc-b84a-867c236f9c2a None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:33.290759 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc3bb70-8ec7-43cc-b84a-867c236f9c2a None admin] PATH_INFO: `/v3/projects` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:33.291915 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc3bb70-8ec7-43cc-b84a-867c236f9c2a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:33.292402 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc3bb70-8ec7-43cc-b84a-867c236f9c2a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:33.294673 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc3bb70-8ec7-43cc-b84a-867c236f9c2a None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:33.318669 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 285/595] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Sep 6 19:59:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:33.319624 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc3bb70-8ec7-43cc-b84a-867c236f9c2a None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:33.381270 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b498a3b-8002-42e0-8b16-471c5c5372dd None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:33.396632 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:33.437807 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375c3f1c-c52c-4770-a0f1-e9f3e40acbb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:33.438482 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375c3f1c-c52c-4770-a0f1-e9f3e40acbb9 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:33.438765 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375c3f1c-c52c-4770-a0f1-e9f3e40acbb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:33.439042 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375c3f1c-c52c-4770-a0f1-e9f3e40acbb9 None admin] PATH_INFO: `/v3/users` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:33.439913 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375c3f1c-c52c-4770-a0f1-e9f3e40acbb9 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:33.440223 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375c3f1c-c52c-4770-a0f1-e9f3e40acbb9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:33.442476 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375c3f1c-c52c-4770-a0f1-e9f3e40acbb9 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:33.465074 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 311/596] 10.222.0.20 () {56 vars in 1139 bytes} [Tue Sep 6 19:59:33 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:33.468069 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375c3f1c-c52c-4770-a0f1-e9f3e40acbb9 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:33.501367 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:33.539646 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc3bb70-8ec7-43cc-b84a-867c236f9c2a None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:33.611661 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3aed94-2e11-4d18-8fbe-d15a6f9acf04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:33.613455 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3aed94-2e11-4d18-8fbe-d15a6f9acf04 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:33.614009 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3aed94-2e11-4d18-8fbe-d15a6f9acf04 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:33.614507 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3aed94-2e11-4d18-8fbe-d15a6f9acf04 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:33.621086 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375c3f1c-c52c-4770-a0f1-e9f3e40acbb9 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:33.637146 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3aed94-2e11-4d18-8fbe-d15a6f9acf04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:33.640332 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3aed94-2e11-4d18-8fbe-d15a6f9acf04 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:33.668873 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957f4725-7ad5-49af-975c-d7e812896883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:33.669953 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957f4725-7ad5-49af-975c-d7e812896883 None admin] REQUEST_METHOD: `PUT` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:33.670388 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957f4725-7ad5-49af-975c-d7e812896883 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:33.670837 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957f4725-7ad5-49af-975c-d7e812896883 None admin] PATH_INFO: `/v3/projects/6aef43493bb1420a9e122313d8db7d5a/users/df1bca993bc142908cc7cf85eabd2b24/roles/b950bf73c93e4f7ebed923ff14e39faa` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:33.671453 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-ca3aed94-2e11-4d18-8fbe-d15a6f9acf04 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 19:59:33.671453 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:33.671453 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:33.671453 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:33.671453 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:33.671453 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:33.671453 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:33.671453 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:33.671453 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:33.671453 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:33.671453 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:33.671453 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:33.671453 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 19:59:33.671453 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 19:59:33.671453 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 19:59:33.671453 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 19:59:33.672385 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:33.672385 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:33.672385 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:33.672385 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:33.672385 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:33.672385 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:33.672385 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:33.672385 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:33.672385 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:33.672385 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:33.672385 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:33.672385 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:33.672385 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:33.672385 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:33.672385 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:33.672385 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:33.672385 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 19:59:33.673304 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 19:59:33.673304 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 19:59:33.673304 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 19:59:33.681869 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 286/597] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 19:59:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 362 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:59:33.683528 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3aed94-2e11-4d18-8fbe-d15a6f9acf04 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:33.693810 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957f4725-7ad5-49af-975c-d7e812896883 None admin] RBAC: Authorizing `identity:create_grant(project_id=6aef43493bb1420a9e122313d8db7d5a, user_id=df1bca993bc142908cc7cf85eabd2b24, role_id=b950bf73c93e4f7ebed923ff14e39faa)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:33.696167 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957f4725-7ad5-49af-975c-d7e812896883 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:33.714842 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 312/598] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Sep 6 19:59:33 2022] PUT /identity/v3/projects/6aef43493bb1420a9e122313d8db7d5a/users/df1bca993bc142908cc7cf85eabd2b24/roles/b950bf73c93e4f7ebed923ff14e39faa => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 19:59:33.716309 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957f4725-7ad5-49af-975c-d7e812896883 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:33.741410 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:33.882076 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957f4725-7ad5-49af-975c-d7e812896883 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:33.921796 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3aed94-2e11-4d18-8fbe-d15a6f9acf04 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:33.949653 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896284bb-d691-4c2b-b66e-965f6dfe99c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:33.952882 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896284bb-d691-4c2b-b66e-965f6dfe99c8 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:33.954633 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896284bb-d691-4c2b-b66e-965f6dfe99c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:33.958774 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896284bb-d691-4c2b-b66e-965f6dfe99c8 None admin] PATH_INFO: `/v3/domains` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:33.959882 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896284bb-d691-4c2b-b66e-965f6dfe99c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:33.960402 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896284bb-d691-4c2b-b66e-965f6dfe99c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:33.963197 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896284bb-d691-4c2b-b66e-965f6dfe99c8 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:33.992903 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 313/599] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 6 19:59:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:34.001961 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0fd6b2-974a-40f2-abf0-c680dc8003ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:34.002752 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896284bb-d691-4c2b-b66e-965f6dfe99c8 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:34.003464 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0fd6b2-974a-40f2-abf0-c680dc8003ee None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:34.004525 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0fd6b2-974a-40f2-abf0-c680dc8003ee None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:34.005085 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0fd6b2-974a-40f2-abf0-c680dc8003ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:34.011910 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:34.039609 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0fd6b2-974a-40f2-abf0-c680dc8003ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:34.040786 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0fd6b2-974a-40f2-abf0-c680dc8003ee None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:34.079547 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-bb0fd6b2-974a-40f2-abf0-c680dc8003ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 19:59:34.079547 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:34.079547 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:34.079547 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:34.079547 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:34.079547 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:34.079547 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:34.079547 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:34.079547 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:34.079547 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:34.079547 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:34.079547 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:34.079547 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 19:59:34.079547 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 19:59:34.079547 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 19:59:34.079547 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 19:59:34.080917 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:34.080917 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:34.080917 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:34.080917 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:34.080917 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:34.080917 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:34.080917 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:34.080917 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:34.080917 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:34.080917 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:34.080917 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:34.080917 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:34.080917 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:34.080917 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:34.080917 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:34.080917 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:34.080917 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 19:59:34.082174 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 19:59:34.082174 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 19:59:34.082174 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 19:59:34.088499 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 287/600] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 19:59:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 400 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:59:34.089897 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0fd6b2-974a-40f2-abf0-c680dc8003ee None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:34.131766 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:34.168299 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896284bb-d691-4c2b-b66e-965f6dfe99c8 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:34.220704 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37de04f-00db-4046-88ed-8065ab059b3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:34.221542 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37de04f-00db-4046-88ed-8065ab059b3d None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:34.222081 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37de04f-00db-4046-88ed-8065ab059b3d None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:34.225203 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37de04f-00db-4046-88ed-8065ab059b3d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:34.248207 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37de04f-00db-4046-88ed-8065ab059b3d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:34.250719 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37de04f-00db-4046-88ed-8065ab059b3d None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:34.258425 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0fd6b2-974a-40f2-abf0-c680dc8003ee None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:34.291488 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-b37de04f-00db-4046-88ed-8065ab059b3d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 19:59:34.291488 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:34.291488 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:34.291488 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:34.291488 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:34.291488 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:34.291488 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:34.291488 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:34.291488 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:34.291488 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:34.291488 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:34.291488 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:34.291488 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 19:59:34.291488 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 19:59:34.291488 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 19:59:34.292655 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 19:59:34.292655 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:34.292655 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:34.292655 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:34.292655 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:34.292655 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:34.292655 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:34.292655 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:34.292655 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:34.292655 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:34.292655 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:34.292655 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:34.292655 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:34.292655 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:34.292655 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:34.292655 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:34.292655 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:34.297986 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 19:59:34.297986 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 19:59:34.297986 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 19:59:34.297986 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 19:59:34.297986 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 19:59:34.297986 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 19:59:34.297986 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 19:59:34.297986 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 19:59:34.297986 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 314/601] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 6 19:59:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 19:59:34.310124 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37de04f-00db-4046-88ed-8065ab059b3d None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:34.322771 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7cd3bb-017a-4377-a3ba-cba41099c310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:34.323679 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7cd3bb-017a-4377-a3ba-cba41099c310 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:34.324146 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7cd3bb-017a-4377-a3ba-cba41099c310 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:34.324576 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7cd3bb-017a-4377-a3ba-cba41099c310 None admin] PATH_INFO: `/v3/domains` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:34.325428 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7cd3bb-017a-4377-a3ba-cba41099c310 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:34.326008 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7cd3bb-017a-4377-a3ba-cba41099c310 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:34.327591 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7cd3bb-017a-4377-a3ba-cba41099c310 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:34.358341 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 288/602] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 6 19:59:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 19:59:34.367156 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7cd3bb-017a-4377-a3ba-cba41099c310 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:34.502286 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37de04f-00db-4046-88ed-8065ab059b3d None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:34.547501 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9015fd9-ddf2-451f-83bf-375eb3f0a144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:34.548392 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9015fd9-ddf2-451f-83bf-375eb3f0a144 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:34.549415 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9015fd9-ddf2-451f-83bf-375eb3f0a144 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:34.549916 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9015fd9-ddf2-451f-83bf-375eb3f0a144 None admin] PATH_INFO: `/v3/projects` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:34.551436 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9015fd9-ddf2-451f-83bf-375eb3f0a144 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:34.551931 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9015fd9-ddf2-451f-83bf-375eb3f0a144 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:34.553579 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9015fd9-ddf2-451f-83bf-375eb3f0a144 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:34.558188 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:34.566811 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:34.579609 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 315/603] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Sep 6 19:59:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 19:59:34.585370 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420910b2-bd85-47ae-8d44-365cc414b99a None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:34.585370 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420910b2-bd85-47ae-8d44-365cc414b99a None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:34.585370 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420910b2-bd85-47ae-8d44-365cc414b99a None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:34.588490 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 316/604] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 19:59:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 19:59:34.589206 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420910b2-bd85-47ae-8d44-365cc414b99a None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:34.662045 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7cd3bb-017a-4377-a3ba-cba41099c310 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:34.708480 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0086fc56-612b-47a2-81d5-ea1a59c2ccd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:34.709581 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0086fc56-612b-47a2-81d5-ea1a59c2ccd3 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:34.710227 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0086fc56-612b-47a2-81d5-ea1a59c2ccd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:34.710722 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0086fc56-612b-47a2-81d5-ea1a59c2ccd3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:34.719298 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0086fc56-612b-47a2-81d5-ea1a59c2ccd3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:34.721278 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0086fc56-612b-47a2-81d5-ea1a59c2ccd3 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:34.731831 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420910b2-bd85-47ae-8d44-365cc414b99a None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:34.735909 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-0086fc56-612b-47a2-81d5-ea1a59c2ccd3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 19:59:34.735909 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:34.735909 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:34.735909 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:34.735909 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:34.735909 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:34.735909 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:34.735909 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:34.735909 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:34.735909 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:34.735909 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:34.735909 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:34.735909 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 19:59:34.735909 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 19:59:34.735909 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 19:59:34.736855 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 19:59:34.736855 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:34.736855 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:34.736855 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:34.736855 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:34.736855 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:34.736855 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:34.736855 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:34.736855 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:34.736855 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:34.736855 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:34.736855 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:34.736855 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:34.736855 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:34.736855 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:34.736855 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:34.736855 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:34.737795 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 19:59:34.737795 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 19:59:34.737795 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 19:59:34.737795 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 19:59:34.737795 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 19:59:34.737795 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 19:59:34.737795 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 19:59:34.737795 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 19:59:34.740532 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 289/605] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 6 19:59:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 374 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:59:34.742266 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acec65ee-5fd6-4a43-8f7d-933fe024eeba None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:34.742773 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acec65ee-5fd6-4a43-8f7d-933fe024eeba None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:34.743250 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acec65ee-5fd6-4a43-8f7d-933fe024eeba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:34.766392 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-acec65ee-5fd6-4a43-8f7d-933fe024eeba None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 19:59:34.768389 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-acec65ee-5fd6-4a43-8f7d-933fe024eeba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 19:59:34.778299 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1f5da3-9fc9-4ff3-a708-1a5c187215b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:34.779590 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1f5da3-9fc9-4ff3-a708-1a5c187215b3 None admin] REQUEST_METHOD: `PUT` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:34.779590 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1f5da3-9fc9-4ff3-a708-1a5c187215b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:34.779817 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1f5da3-9fc9-4ff3-a708-1a5c187215b3 None admin] PATH_INFO: `/v3/projects/6aef43493bb1420a9e122313d8db7d5a/users/5776cbf4f6714b2083c3ebdd6d98cc87/roles/b950bf73c93e4f7ebed923ff14e39faa` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:34.800128 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1f5da3-9fc9-4ff3-a708-1a5c187215b3 None admin] RBAC: Authorizing `identity:create_grant(project_id=6aef43493bb1420a9e122313d8db7d5a, user_id=5776cbf4f6714b2083c3ebdd6d98cc87, role_id=b950bf73c93e4f7ebed923ff14e39faa)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:34.802062 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1f5da3-9fc9-4ff3-a708-1a5c187215b3 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:34.827644 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 317/606] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Sep 6 19:59:34 2022] PUT /identity/v3/projects/6aef43493bb1420a9e122313d8db7d5a/users/5776cbf4f6714b2083c3ebdd6d98cc87/roles/b950bf73c93e4f7ebed923ff14e39faa => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 19:59:34.827644 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1f5da3-9fc9-4ff3-a708-1a5c187215b3 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:34.888412 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:34.899169 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 290/607] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 19:59:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 19:59:35.003503 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1f5da3-9fc9-4ff3-a708-1a5c187215b3 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:35.075988 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d3db48-c3b2-40c6-9e05-b199f7e61233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:35.076694 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d3db48-c3b2-40c6-9e05-b199f7e61233 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:35.077075 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d3db48-c3b2-40c6-9e05-b199f7e61233 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:35.077360 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d3db48-c3b2-40c6-9e05-b199f7e61233 None admin] PATH_INFO: `/v3/projects` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:35.078218 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d3db48-c3b2-40c6-9e05-b199f7e61233 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:35.078532 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d3db48-c3b2-40c6-9e05-b199f7e61233 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:35.080259 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d3db48-c3b2-40c6-9e05-b199f7e61233 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:35.095874 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 318/608] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Sep 6 19:59:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:35.102334 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acec65ee-5fd6-4a43-8f7d-933fe024eeba None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:35.155951 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:35.265318 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acec65ee-5fd6-4a43-8f7d-933fe024eeba None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:35.321989 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c87e59-9ae1-4aa2-9c8a-d2c5985c614e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:35.322741 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c87e59-9ae1-4aa2-9c8a-d2c5985c614e None admin] REQUEST_METHOD: `PUT` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:35.323029 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c87e59-9ae1-4aa2-9c8a-d2c5985c614e None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:35.323301 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c87e59-9ae1-4aa2-9c8a-d2c5985c614e None admin] PATH_INFO: `/v3/projects/6aef43493bb1420a9e122313d8db7d5a/users/32882d38295046fea004a659a71bde58/roles/b950bf73c93e4f7ebed923ff14e39faa` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:35.352156 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c87e59-9ae1-4aa2-9c8a-d2c5985c614e None admin] RBAC: Authorizing `identity:create_grant(project_id=6aef43493bb1420a9e122313d8db7d5a, user_id=32882d38295046fea004a659a71bde58, role_id=b950bf73c93e4f7ebed923ff14e39faa)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:35.355613 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c87e59-9ae1-4aa2-9c8a-d2c5985c614e None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:35.379549 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 291/609] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Sep 6 19:59:35 2022] PUT /identity/v3/projects/6aef43493bb1420a9e122313d8db7d5a/users/32882d38295046fea004a659a71bde58/roles/b950bf73c93e4f7ebed923ff14e39faa => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 19:59:35.500055 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d3db48-c3b2-40c6-9e05-b199f7e61233 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:35.641386 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d3db48-c3b2-40c6-9e05-b199f7e61233 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:35.656803 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:35.694581 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2856399c-ba38-4868-87fa-0eb27e6feeb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:35.695425 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2856399c-ba38-4868-87fa-0eb27e6feeb8 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:35.695710 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2856399c-ba38-4868-87fa-0eb27e6feeb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:35.695995 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2856399c-ba38-4868-87fa-0eb27e6feeb8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:35.710232 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2856399c-ba38-4868-87fa-0eb27e6feeb8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 19:59:35.716273 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-2856399c-ba38-4868-87fa-0eb27e6feeb8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 19:59:35.716273 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:35.716273 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:35.716273 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:35.716273 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:35.716273 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:35.716273 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:35.716273 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:35.716273 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:35.716273 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:35.716273 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:35.716273 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:35.716273 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 19:59:35.716273 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 19:59:35.716273 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 19:59:35.716273 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 19:59:35.717379 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 19:59:35.717379 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 19:59:35.717379 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 19:59:35.717379 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 19:59:35.717379 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:35.717379 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:35.717379 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:35.717379 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:35.717379 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:35.717379 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:35.717379 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:35.717379 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:35.717379 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:35.717379 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:35.717379 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:35.717379 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:35.718435 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:35.718435 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:35.718435 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:35.718435 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:35.718435 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 19:59:35.718435 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 19:59:35.718435 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 19:59:35.718435 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 19:59:35.718435 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 19:59:35.718435 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 19:59:35.718435 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 19:59:35.718435 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 19:59:35.718435 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 319/610] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Sep 6 19:59:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:59:35.725029 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c87e59-9ae1-4aa2-9c8a-d2c5985c614e None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:35.859523 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:35.885408 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c87e59-9ae1-4aa2-9c8a-d2c5985c614e None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:35.980391 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2663d13e-3fd3-402c-b38a-61324e042888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:35.981246 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2663d13e-3fd3-402c-b38a-61324e042888 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:35.981698 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2663d13e-3fd3-402c-b38a-61324e042888 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:35.982121 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2663d13e-3fd3-402c-b38a-61324e042888 None admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:35.983213 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2663d13e-3fd3-402c-b38a-61324e042888 None admin] RBAC: Adding query filter params (name=service) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:35.983691 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2663d13e-3fd3-402c-b38a-61324e042888 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:35.986022 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2663d13e-3fd3-402c-b38a-61324e042888 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:36.004846 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 292/611] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Sep 6 19:59:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:36.011245 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2856399c-ba38-4868-87fa-0eb27e6feeb8 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:36.176812 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2856399c-ba38-4868-87fa-0eb27e6feeb8 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:36.238577 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d7b92e-6074-4e8a-8a49-4b20b99d6dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:36.239166 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d7b92e-6074-4e8a-8a49-4b20b99d6dd1 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:36.239453 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d7b92e-6074-4e8a-8a49-4b20b99d6dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:36.239736 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d7b92e-6074-4e8a-8a49-4b20b99d6dd1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:36.259199 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d7b92e-6074-4e8a-8a49-4b20b99d6dd1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:36.260851 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d7b92e-6074-4e8a-8a49-4b20b99d6dd1 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:36.287101 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22478282-0521-4242-ad2d-538ae3461411 None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:36.288062 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22478282-0521-4242-ad2d-538ae3461411 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:36.288175 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-71d7b92e-6074-4e8a-8a49-4b20b99d6dd1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 19:59:36.288175 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:36.288175 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:36.288175 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:36.288175 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:36.288175 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:36.288175 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:36.288175 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:36.288175 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:36.288175 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:36.288175 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:36.288175 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:36.288175 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 19:59:36.288175 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 19:59:36.288175 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 19:59:36.288175 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 19:59:36.289065 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:36.289065 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:36.289065 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:36.289065 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:36.289065 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:36.289065 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:36.289065 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:36.289065 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:36.289065 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:36.289065 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:36.289065 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:36.289065 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:36.289065 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:36.289065 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:36.289065 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:36.289065 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:36.289065 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 19:59:36.290013 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 19:59:36.290013 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 19:59:36.290013 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 19:59:36.290013 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 320/612] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 19:59:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:59:36.291526 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22478282-0521-4242-ad2d-538ae3461411 None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:36.293396 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 293/613] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 19:59:36 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 19:59:36.299500 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d7b92e-6074-4e8a-8a49-4b20b99d6dd1 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:36.306319 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4dc54d-a739-458f-82b1-e7c1cdce8460 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:36.306965 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4dc54d-a739-458f-82b1-e7c1cdce8460 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:36.307466 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4dc54d-a739-458f-82b1-e7c1cdce8460 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:36.342024 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-9e4dc54d-a739-458f-82b1-e7c1cdce8460 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 19:59:36.343350 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-9e4dc54d-a739-458f-82b1-e7c1cdce8460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 19:59:36.350041 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:36.523370 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d7b92e-6074-4e8a-8a49-4b20b99d6dd1 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:36.540372 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 294/614] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 19:59:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 19:59:36.645727 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4054bd41-133e-4c2d-a1c6-52df16107350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:36.646015 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4054bd41-133e-4c2d-a1c6-52df16107350 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:36.646590 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4054bd41-133e-4c2d-a1c6-52df16107350 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:36.647883 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4054bd41-133e-4c2d-a1c6-52df16107350 None admin] PATH_INFO: `/v3/domains` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:36.653662 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4054bd41-133e-4c2d-a1c6-52df16107350 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:36.653662 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4054bd41-133e-4c2d-a1c6-52df16107350 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:36.653662 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4054bd41-133e-4c2d-a1c6-52df16107350 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:36.676933 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 321/615] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 6 19:59:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 378 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:36.686445 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4dc54d-a739-458f-82b1-e7c1cdce8460 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:36.884423 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:36.938952 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4dc54d-a739-458f-82b1-e7c1cdce8460 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:37.027433 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff2783b-27b0-4e93-8dac-d8b18165a0fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:37.029130 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff2783b-27b0-4e93-8dac-d8b18165a0fe None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:37.030091 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff2783b-27b0-4e93-8dac-d8b18165a0fe None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:37.030898 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff2783b-27b0-4e93-8dac-d8b18165a0fe None admin] PATH_INFO: `/v3/users/swift` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:37.051017 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff2783b-27b0-4e93-8dac-d8b18165a0fe None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:37.054263 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff2783b-27b0-4e93-8dac-d8b18165a0fe None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:37.079227 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-4ff2783b-27b0-4e93-8dac-d8b18165a0fe None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 06 19:59:37.079227 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:37.079227 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:37.079227 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:37.079227 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:37.079227 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:37.079227 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:37.079227 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:37.079227 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:37.079227 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:37.079227 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:37.079227 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:37.079227 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 19:59:37.079227 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 19:59:37.079227 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 19:59:37.079227 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 19:59:37.081099 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:37.081099 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:37.081099 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 19:59:37.081099 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 19:59:37.081099 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 19:59:37.081099 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 19:59:37.081099 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:37.081099 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:37.081099 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:37.081099 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:37.081099 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:37.081099 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:37.081099 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:37.081099 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:37.081099 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:37.081099 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:37.081099 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:37.081099 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:37.083414 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:37.083414 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:37.083414 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 19:59:37.083414 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 19:59:37.083414 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 19:59:37.083414 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 19:59:37.083414 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 19:59:37.083414 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 19:59:37.083414 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 06 19:59:37.083414 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 19:59:37.088112 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 295/616] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Sep 6 19:59:36 2022] GET /identity/v3/users/swift => generated 83 bytes in 403 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:59:37.098382 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4054bd41-133e-4c2d-a1c6-52df16107350 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:37.264335 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:37.269510 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff2783b-27b0-4e93-8dac-d8b18165a0fe None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:37.331338 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4054bd41-133e-4c2d-a1c6-52df16107350 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:37.407924 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4d67ef-3c12-401d-9338-2448ca239d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:37.412520 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4d67ef-3c12-401d-9338-2448ca239d05 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:37.413125 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4d67ef-3c12-401d-9338-2448ca239d05 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:37.413524 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4d67ef-3c12-401d-9338-2448ca239d05 None admin] PATH_INFO: `/v3/users` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:37.414907 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4d67ef-3c12-401d-9338-2448ca239d05 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:37.415443 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4d67ef-3c12-401d-9338-2448ca239d05 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:37.418263 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4d67ef-3c12-401d-9338-2448ca239d05 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:37.484251 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 322/617] 10.222.0.20 () {56 vars in 1135 bytes} [Tue Sep 6 19:59:37 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 387 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:37.493310 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4d67ef-3c12-401d-9338-2448ca239d05 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:37.631771 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff2783b-27b0-4e93-8dac-d8b18165a0fe None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:37.718732 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633a198f-81dc-4c4f-a1f7-c017b0f0c725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:37.720868 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633a198f-81dc-4c4f-a1f7-c017b0f0c725 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:37.721868 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633a198f-81dc-4c4f-a1f7-c017b0f0c725 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:37.722510 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633a198f-81dc-4c4f-a1f7-c017b0f0c725 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:37.738396 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:37.745573 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4d67ef-3c12-401d-9338-2448ca239d05 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:37.749512 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633a198f-81dc-4c4f-a1f7-c017b0f0c725 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 19:59:37.759924 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-633a198f-81dc-4c4f-a1f7-c017b0f0c725 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 19:59:37.759924 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:37.759924 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:37.759924 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:37.759924 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:37.759924 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:37.759924 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:37.759924 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:37.759924 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:37.759924 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:37.759924 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:37.759924 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:37.759924 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 19:59:37.759924 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 19:59:37.759924 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 19:59:37.759924 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 19:59:37.762247 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 19:59:37.762247 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 19:59:37.762247 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 19:59:37.762247 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 19:59:37.762247 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:37.762247 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:37.762247 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:37.762247 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:37.762247 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:37.762247 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:37.762247 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:37.762247 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:37.762247 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:37.762247 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:37.762247 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:37.762247 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:37.764200 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:37.764200 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:37.764200 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:37.764200 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:37.764200 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 19:59:37.764200 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 19:59:37.764200 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 19:59:37.764200 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 19:59:37.764200 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 19:59:37.764200 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 19:59:37.764200 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 19:59:37.764200 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 19:59:37.770655 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 296/618] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Sep 6 19:59:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 501 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 19:59:37.772196 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce6b7cd-b4ce-4def-92b5-be9ae87ad127 None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:37.772704 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce6b7cd-b4ce-4def-92b5-be9ae87ad127 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:37.773200 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce6b7cd-b4ce-4def-92b5-be9ae87ad127 None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:37.783506 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 297/619] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 19:59:37 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 19:59:37.786039 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce6b7cd-b4ce-4def-92b5-be9ae87ad127 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:37.825241 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fa12b0-9134-407c-9e9c-0894b021e7ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:37.826912 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fa12b0-9134-407c-9e9c-0894b021e7ef None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:37.827603 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fa12b0-9134-407c-9e9c-0894b021e7ef None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:37.828295 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fa12b0-9134-407c-9e9c-0894b021e7ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:37.837106 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:37.853656 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fa12b0-9134-407c-9e9c-0894b021e7ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:37.856338 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fa12b0-9134-407c-9e9c-0894b021e7ef None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:37.884094 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-65fa12b0-9134-407c-9e9c-0894b021e7ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 19:59:37.884094 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:37.884094 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:37.884094 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:37.884094 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:37.884094 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:37.884094 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:37.884094 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:37.884094 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:37.884094 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:37.884094 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:37.884094 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:37.884094 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 19:59:37.884094 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 19:59:37.884094 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 19:59:37.884094 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 19:59:37.886015 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:37.886015 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:37.886015 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:37.886015 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:37.886015 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:37.886015 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:37.886015 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:37.886015 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:37.886015 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:37.886015 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:37.886015 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:37.886015 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:37.886015 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:37.886015 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:37.886015 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:37.886015 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:37.886015 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 19:59:37.888587 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 19:59:37.888587 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 19:59:37.888587 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 19:59:37.893799 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 323/620] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 19:59:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 394 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 19:59:37.896357 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086561d7-3b51-40cd-915c-7a2bdae4d4d7 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:37.896848 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086561d7-3b51-40cd-915c-7a2bdae4d4d7 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:37.897352 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086561d7-3b51-40cd-915c-7a2bdae4d4d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:37.927876 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-086561d7-3b51-40cd-915c-7a2bdae4d4d7 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 19:59:37.929748 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-086561d7-3b51-40cd-915c-7a2bdae4d4d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 19:59:38.020555 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce6b7cd-b4ce-4def-92b5-be9ae87ad127 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:38.058766 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 324/621] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 19:59:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 19:59:38.060111 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086561d7-3b51-40cd-915c-7a2bdae4d4d7 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:38.068718 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8719c5ce-9608-4017-b154-fcec5d901484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:38.069974 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8719c5ce-9608-4017-b154-fcec5d901484 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:38.070531 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8719c5ce-9608-4017-b154-fcec5d901484 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:38.071050 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8719c5ce-9608-4017-b154-fcec5d901484 None admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:38.071957 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8719c5ce-9608-4017-b154-fcec5d901484 None admin] RBAC: Adding query filter params (name=service) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:38.072539 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8719c5ce-9608-4017-b154-fcec5d901484 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:38.074283 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8719c5ce-9608-4017-b154-fcec5d901484 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:38.092446 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 298/622] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Sep 6 19:59:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:38.103742 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8719c5ce-9608-4017-b154-fcec5d901484 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:38.225334 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:38.268844 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086561d7-3b51-40cd-915c-7a2bdae4d4d7 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:38.282798 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8719c5ce-9608-4017-b154-fcec5d901484 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:38.308478 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:38.331979 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667d0a26-1b09-4a07-8d5c-3853c81dcdec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:38.334375 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667d0a26-1b09-4a07-8d5c-3853c81dcdec None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:38.335284 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667d0a26-1b09-4a07-8d5c-3853c81dcdec None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:38.336115 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667d0a26-1b09-4a07-8d5c-3853c81dcdec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:38.343647 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec59c87-8641-41be-a35c-b381d444ab88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:38.344611 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec59c87-8641-41be-a35c-b381d444ab88 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:38.345092 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec59c87-8641-41be-a35c-b381d444ab88 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:38.345556 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec59c87-8641-41be-a35c-b381d444ab88 None admin] PATH_INFO: `/v3/domains` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:38.346810 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec59c87-8641-41be-a35c-b381d444ab88 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:38.347352 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec59c87-8641-41be-a35c-b381d444ab88 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:38.350926 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec59c87-8641-41be-a35c-b381d444ab88 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:38.353679 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667d0a26-1b09-4a07-8d5c-3853c81dcdec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:38.355860 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667d0a26-1b09-4a07-8d5c-3853c81dcdec None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:38.374947 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-667d0a26-1b09-4a07-8d5c-3853c81dcdec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 19:59:38.374947 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:38.374947 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:38.374947 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:38.374947 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:38.374947 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:38.374947 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:38.374947 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:38.374947 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:38.374947 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:38.374947 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:38.374947 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:38.374947 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 19:59:38.374947 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 19:59:38.374947 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 19:59:38.374947 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 19:59:38.376139 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:38.376139 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:38.376139 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:38.376139 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:38.376139 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:38.376139 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:38.376139 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:38.376139 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:38.376139 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:38.376139 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:38.376139 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:38.376139 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:38.376139 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:38.376139 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:38.376139 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:38.376139 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:38.376139 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 19:59:38.377115 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 19:59:38.377115 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 19:59:38.377115 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 19:59:38.378621 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 299/623] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 19:59:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:59:38.382890 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6f9fa6-2049-4501-b8a0-217cd8f2b50b None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:38.382890 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6f9fa6-2049-4501-b8a0-217cd8f2b50b None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:38.382890 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6f9fa6-2049-4501-b8a0-217cd8f2b50b None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:38.385739 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 300/624] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 19:59:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 19:59:38.387145 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6f9fa6-2049-4501-b8a0-217cd8f2b50b None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:38.397006 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 325/625] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 6 19:59:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:38.399877 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deee2315-d512-490e-8da5-b29cb9774a65 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:38.400505 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deee2315-d512-490e-8da5-b29cb9774a65 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:38.400996 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deee2315-d512-490e-8da5-b29cb9774a65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:38.432737 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-deee2315-d512-490e-8da5-b29cb9774a65 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 19:59:38.434886 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-deee2315-d512-490e-8da5-b29cb9774a65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 19:59:38.534965 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 326/626] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 19:59:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 19:59:38.537382 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deee2315-d512-490e-8da5-b29cb9774a65 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:38.578698 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6f9fa6-2049-4501-b8a0-217cd8f2b50b None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:38.613480 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:38.659323 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31a2493-1a82-440e-a09f-a86381bd4c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:38.660022 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31a2493-1a82-440e-a09f-a86381bd4c3f None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:38.660244 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31a2493-1a82-440e-a09f-a86381bd4c3f None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:38.661130 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31a2493-1a82-440e-a09f-a86381bd4c3f None admin] PATH_INFO: `/v3/domains` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:38.662537 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31a2493-1a82-440e-a09f-a86381bd4c3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:38.662866 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31a2493-1a82-440e-a09f-a86381bd4c3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:38.664315 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31a2493-1a82-440e-a09f-a86381bd4c3f None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:38.689650 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 301/627] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 6 19:59:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:38.689650 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31a2493-1a82-440e-a09f-a86381bd4c3f None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:38.782992 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deee2315-d512-490e-8da5-b29cb9774a65 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:38.800376 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:38.838216 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64c88cc-8c13-4d4f-bc2c-059e012258da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:38.838868 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64c88cc-8c13-4d4f-bc2c-059e012258da None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:38.839150 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64c88cc-8c13-4d4f-bc2c-059e012258da None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:38.839488 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64c88cc-8c13-4d4f-bc2c-059e012258da None admin] PATH_INFO: `/v3/roles/service` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:38.852645 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64c88cc-8c13-4d4f-bc2c-059e012258da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 19:59:38.854391 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31a2493-1a82-440e-a09f-a86381bd4c3f None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:38.857960 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-c64c88cc-8c13-4d4f-bc2c-059e012258da None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 19:59:38.857960 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:38.857960 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:38.857960 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:38.857960 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:38.857960 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:38.857960 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:38.857960 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:38.857960 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:38.857960 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:38.857960 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:38.857960 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:38.857960 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 19:59:38.857960 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 19:59:38.857960 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 19:59:38.857960 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 19:59:38.859781 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 19:59:38.859781 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 19:59:38.859781 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 19:59:38.859781 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 19:59:38.859781 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:38.859781 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:38.859781 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:38.859781 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:38.859781 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:38.859781 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:38.859781 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:38.859781 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:38.859781 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:38.859781 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:38.859781 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:38.859781 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:38.862017 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:38.862017 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:38.862017 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:38.862017 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:38.862017 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 19:59:38.862017 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 19:59:38.862017 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 19:59:38.862017 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 19:59:38.862017 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 19:59:38.862017 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 19:59:38.862017 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 19:59:38.862017 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 19:59:38.862017 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 327/628] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Sep 6 19:59:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:59:38.862017 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64c88cc-8c13-4d4f-bc2c-059e012258da None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:38.910836 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eb9568-4b13-4855-adac-fc84e8dc610a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:38.911413 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eb9568-4b13-4855-adac-fc84e8dc610a None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:38.911694 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eb9568-4b13-4855-adac-fc84e8dc610a None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:38.911964 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eb9568-4b13-4855-adac-fc84e8dc610a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:38.920619 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4eb9568-4b13-4855-adac-fc84e8dc610a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:38.922200 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4eb9568-4b13-4855-adac-fc84e8dc610a None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:38.935770 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-d4eb9568-4b13-4855-adac-fc84e8dc610a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 19:59:38.935770 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:38.935770 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:38.935770 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:38.935770 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:38.935770 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:38.935770 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:38.935770 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:38.935770 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:38.935770 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:38.935770 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:38.935770 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:38.935770 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 19:59:38.935770 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 19:59:38.935770 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 19:59:38.937393 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 19:59:38.937393 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:38.937393 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:38.937393 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:38.937393 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:38.937393 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:38.937393 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:38.937393 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:38.937393 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:38.937393 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:38.937393 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:38.937393 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:38.937393 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:38.937393 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:38.937393 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:38.937393 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:38.937393 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:38.939271 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 19:59:38.939271 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 19:59:38.939271 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 19:59:38.939271 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 19:59:38.939271 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 19:59:38.939271 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 19:59:38.939271 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 19:59:38.939271 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 19:59:38.939271 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 302/629] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 6 19:59:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:59:38.939271 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d65789-206b-487c-a751-173a1269b885 None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:38.939271 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d65789-206b-487c-a751-173a1269b885 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:38.939924 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d65789-206b-487c-a751-173a1269b885 None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:38.940365 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 303/630] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 19:59:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 19:59:38.941136 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d65789-206b-487c-a751-173a1269b885 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:38.979781 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:39.010686 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64c88cc-8c13-4d4f-bc2c-059e012258da None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:39.054843 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066c81bf-7051-4ad8-95dc-f3590329fe3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:39.055517 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066c81bf-7051-4ad8-95dc-f3590329fe3f None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:39.055887 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066c81bf-7051-4ad8-95dc-f3590329fe3f None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:39.056165 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066c81bf-7051-4ad8-95dc-f3590329fe3f None admin] PATH_INFO: `/v3/users/glance` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:39.064772 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066c81bf-7051-4ad8-95dc-f3590329fe3f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:39.066356 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066c81bf-7051-4ad8-95dc-f3590329fe3f None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:39.072120 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d65789-206b-487c-a751-173a1269b885 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:39.081477 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-066c81bf-7051-4ad8-95dc-f3590329fe3f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 06 19:59:39.081477 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:39.081477 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:39.081477 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:39.081477 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:39.081477 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:39.081477 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:39.081477 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:39.081477 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:39.081477 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:39.081477 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:39.081477 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:39.081477 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 19:59:39.081477 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 19:59:39.081477 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 19:59:39.081477 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 19:59:39.083934 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:39.083934 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:39.083934 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 19:59:39.083934 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 19:59:39.083934 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 19:59:39.083934 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 19:59:39.083934 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:39.083934 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:39.083934 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:39.083934 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:39.083934 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:39.083934 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:39.083934 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:39.083934 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:39.083934 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:39.083934 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:39.083934 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:39.083934 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:39.086950 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:39.086950 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:39.086950 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 19:59:39.086950 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 19:59:39.086950 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 19:59:39.086950 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 19:59:39.086950 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 19:59:39.086950 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 19:59:39.086950 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 06 19:59:39.086950 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 19:59:39.086950 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 328/631] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 6 19:59:38 2022] GET /identity/v3/users/glance => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:59:39.086950 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066c81bf-7051-4ad8-95dc-f3590329fe3f None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:39.124336 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab940a4-35df-4f72-8904-b344bb10f4e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:39.125444 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab940a4-35df-4f72-8904-b344bb10f4e2 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:39.125974 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab940a4-35df-4f72-8904-b344bb10f4e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:39.126230 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab940a4-35df-4f72-8904-b344bb10f4e2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:39.129267 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:39.141840 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab940a4-35df-4f72-8904-b344bb10f4e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 19:59:39.152396 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-4ab940a4-35df-4f72-8904-b344bb10f4e2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 19:59:39.152396 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:39.152396 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:39.152396 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:39.152396 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:39.152396 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:39.152396 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:39.152396 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:39.152396 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:39.152396 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:39.152396 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:39.152396 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:39.152396 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 19:59:39.152396 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 19:59:39.152396 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 19:59:39.152396 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 19:59:39.154518 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 19:59:39.154518 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 19:59:39.154518 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 19:59:39.154518 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 19:59:39.154518 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:39.154518 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:39.154518 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:39.154518 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:39.154518 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:39.154518 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:39.154518 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:39.154518 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:39.154518 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:39.154518 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:39.154518 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:39.154518 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:39.156086 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:39.156086 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:39.156086 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:39.156086 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:39.156086 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 19:59:39.156086 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 19:59:39.156086 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 19:59:39.156086 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 19:59:39.156086 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 19:59:39.156086 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 19:59:39.156086 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 19:59:39.156086 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 19:59:39.156086 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 304/632] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Sep 6 19:59:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:59:39.157642 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab940a4-35df-4f72-8904-b344bb10f4e2 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:39.233263 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066c81bf-7051-4ad8-95dc-f3590329fe3f None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:39.277585 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4249f523-5037-4295-97ee-5e5f3d1e5ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:39.278610 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4249f523-5037-4295-97ee-5e5f3d1e5ea7 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:39.278610 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4249f523-5037-4295-97ee-5e5f3d1e5ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:39.278890 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4249f523-5037-4295-97ee-5e5f3d1e5ea7 None admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:39.279437 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4249f523-5037-4295-97ee-5e5f3d1e5ea7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:39.279742 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4249f523-5037-4295-97ee-5e5f3d1e5ea7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:39.281164 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4249f523-5037-4295-97ee-5e5f3d1e5ea7 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:39.291013 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 329/633] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Sep 6 19:59:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:39.292400 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15b76d5-71a0-471e-ba55-6f707beed2d9 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:39.292837 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15b76d5-71a0-471e-ba55-6f707beed2d9 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:39.293119 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15b76d5-71a0-471e-ba55-6f707beed2d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:39.294236 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:39.310731 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab940a4-35df-4f72-8904-b344bb10f4e2 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:39.314417 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-a15b76d5-71a0-471e-ba55-6f707beed2d9 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 19:59:39.315659 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-a15b76d5-71a0-471e-ba55-6f707beed2d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 19:59:39.369181 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9537b0d6-0659-450e-929f-506575e08f18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:39.369841 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9537b0d6-0659-450e-929f-506575e08f18 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:39.370124 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9537b0d6-0659-450e-929f-506575e08f18 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:39.370401 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9537b0d6-0659-450e-929f-506575e08f18 None admin] PATH_INFO: `/v3/projects` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:39.371125 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9537b0d6-0659-450e-929f-506575e08f18 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:39.371435 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9537b0d6-0659-450e-929f-506575e08f18 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:39.372911 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9537b0d6-0659-450e-929f-506575e08f18 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:39.387713 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 305/634] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Sep 6 19:59:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:39.389855 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9537b0d6-0659-450e-929f-506575e08f18 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:39.407109 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 330/635] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 19:59:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 19:59:39.407849 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15b76d5-71a0-471e-ba55-6f707beed2d9 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:39.445034 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:39.558518 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15b76d5-71a0-471e-ba55-6f707beed2d9 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:39.581467 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9537b0d6-0659-450e-929f-506575e08f18 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:39.617478 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42460121-dd3f-49a0-998c-814f90a4c9f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:39.618609 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42460121-dd3f-49a0-998c-814f90a4c9f5 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:39.618908 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42460121-dd3f-49a0-998c-814f90a4c9f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:39.619348 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42460121-dd3f-49a0-998c-814f90a4c9f5 None admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:39.621321 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42460121-dd3f-49a0-998c-814f90a4c9f5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:39.621856 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42460121-dd3f-49a0-998c-814f90a4c9f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:39.624193 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42460121-dd3f-49a0-998c-814f90a4c9f5 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:39.647962 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 331/636] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Sep 6 19:59:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:39.649428 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e57e74-537b-4354-aac6-d97de4286fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:39.649653 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42460121-dd3f-49a0-998c-814f90a4c9f5 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:39.650664 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e57e74-537b-4354-aac6-d97de4286fe3 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:39.650996 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e57e74-537b-4354-aac6-d97de4286fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:39.651374 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e57e74-537b-4354-aac6-d97de4286fe3 None admin] PATH_INFO: `/v3/users` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:39.652285 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e57e74-537b-4354-aac6-d97de4286fe3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:39.652692 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e57e74-537b-4354-aac6-d97de4286fe3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:39.656673 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e57e74-537b-4354-aac6-d97de4286fe3 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:39.687492 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 306/637] 10.222.0.20 () {56 vars in 1137 bytes} [Tue Sep 6 19:59:39 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:39.689009 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e57e74-537b-4354-aac6-d97de4286fe3 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:39.704670 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:39.786156 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:39.809654 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42460121-dd3f-49a0-998c-814f90a4c9f5 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:39.847226 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e57e74-537b-4354-aac6-d97de4286fe3 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:39.859343 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2f2c04-f053-4962-89c5-a60485816bd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:39.860406 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2f2c04-f053-4962-89c5-a60485816bd8 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:39.860910 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2f2c04-f053-4962-89c5-a60485816bd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:39.861409 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2f2c04-f053-4962-89c5-a60485816bd8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:39.878920 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2f2c04-f053-4962-89c5-a60485816bd8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:39.880619 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2f2c04-f053-4962-89c5-a60485816bd8 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:39.898618 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7ff826-21cd-47da-a715-93a1f2229ba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:39.899162 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7ff826-21cd-47da-a715-93a1f2229ba0 None admin] REQUEST_METHOD: `PUT` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:39.899476 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7ff826-21cd-47da-a715-93a1f2229ba0 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:39.899720 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7ff826-21cd-47da-a715-93a1f2229ba0 None admin] PATH_INFO: `/v3/projects/6aef43493bb1420a9e122313d8db7d5a/users/2b72516236da47e292d78de201b0552b/roles/b950bf73c93e4f7ebed923ff14e39faa` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:39.902799 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-fc2f2c04-f053-4962-89c5-a60485816bd8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 19:59:39.902799 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:39.902799 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:39.902799 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:39.902799 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:39.902799 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:39.902799 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:39.902799 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:39.902799 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:39.902799 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:39.902799 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:39.902799 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:39.902799 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 19:59:39.902799 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 19:59:39.902799 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 19:59:39.902799 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 19:59:39.904407 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:39.904407 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:39.904407 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:39.904407 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:39.904407 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:39.904407 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:39.904407 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:39.904407 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:39.904407 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:39.904407 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:39.904407 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:39.904407 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:39.904407 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:39.904407 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:39.904407 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:39.904407 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:39.904407 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 19:59:39.906717 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 19:59:39.906717 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 19:59:39.906717 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 19:59:39.906717 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 332/638] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 19:59:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:59:39.906717 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2f2c04-f053-4962-89c5-a60485816bd8 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:39.930096 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7ff826-21cd-47da-a715-93a1f2229ba0 None admin] RBAC: Authorizing `identity:create_grant(project_id=6aef43493bb1420a9e122313d8db7d5a, user_id=2b72516236da47e292d78de201b0552b, role_id=b950bf73c93e4f7ebed923ff14e39faa)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:39.932503 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7ff826-21cd-47da-a715-93a1f2229ba0 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:39.952465 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 307/639] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Sep 6 19:59:39 2022] PUT /identity/v3/projects/6aef43493bb1420a9e122313d8db7d5a/users/2b72516236da47e292d78de201b0552b/roles/b950bf73c93e4f7ebed923ff14e39faa => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 19:59:39.953922 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7ff826-21cd-47da-a715-93a1f2229ba0 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:40.075598 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:40.093212 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2f2c04-f053-4962-89c5-a60485816bd8 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:40.144166 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53b419e-666b-46f2-b4b7-944ef65a0d6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:40.144846 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53b419e-666b-46f2-b4b7-944ef65a0d6b None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:40.145199 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53b419e-666b-46f2-b4b7-944ef65a0d6b None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:40.145488 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53b419e-666b-46f2-b4b7-944ef65a0d6b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:40.148086 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:40.160332 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7ff826-21cd-47da-a715-93a1f2229ba0 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:40.162387 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53b419e-666b-46f2-b4b7-944ef65a0d6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:40.164375 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53b419e-666b-46f2-b4b7-944ef65a0d6b None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:40.190393 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-d53b419e-666b-46f2-b4b7-944ef65a0d6b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 19:59:40.190393 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:40.190393 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:40.190393 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:40.190393 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:40.190393 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:40.190393 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:40.190393 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:40.190393 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:40.190393 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:40.190393 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:40.190393 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:40.190393 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 19:59:40.190393 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 19:59:40.190393 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 19:59:40.190393 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 19:59:40.191487 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:40.191487 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:40.191487 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:40.191487 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:40.191487 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:40.191487 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:40.191487 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:40.191487 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:40.191487 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:40.191487 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:40.191487 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:40.191487 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:40.191487 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:40.191487 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:40.191487 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:40.191487 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:40.191487 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 19:59:40.192985 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 19:59:40.192985 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 19:59:40.192985 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 19:59:40.192985 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 333/640] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 19:59:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:59:40.199398 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53b419e-666b-46f2-b4b7-944ef65a0d6b None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:40.228681 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aecc23-36de-4f4f-b38d-e6dfcc2f9185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:40.229782 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aecc23-36de-4f4f-b38d-e6dfcc2f9185 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:40.230238 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aecc23-36de-4f4f-b38d-e6dfcc2f9185 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:40.230698 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aecc23-36de-4f4f-b38d-e6dfcc2f9185 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:40.359713 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53b419e-666b-46f2-b4b7-944ef65a0d6b None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:40.369232 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8aecc23-36de-4f4f-b38d-e6dfcc2f9185 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:40.371027 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8aecc23-36de-4f4f-b38d-e6dfcc2f9185 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:40.411683 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4905ec-9141-4cc7-88eb-4e90655dd197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:40.412776 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4905ec-9141-4cc7-88eb-4e90655dd197 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:40.413066 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4905ec-9141-4cc7-88eb-4e90655dd197 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:40.413349 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4905ec-9141-4cc7-88eb-4e90655dd197 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:40.425881 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4905ec-9141-4cc7-88eb-4e90655dd197 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 19:59:40.432240 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-5d4905ec-9141-4cc7-88eb-4e90655dd197 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 19:59:40.432240 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:40.432240 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:40.432240 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:40.432240 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:40.432240 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:40.432240 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:40.432240 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:40.432240 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:40.432240 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:40.432240 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:40.432240 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:40.432240 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 19:59:40.432240 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 19:59:40.432240 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 19:59:40.432240 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 19:59:40.434309 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 19:59:40.434309 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 19:59:40.434309 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 19:59:40.434309 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 19:59:40.434309 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:40.434309 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:40.434309 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:40.434309 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:40.434309 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:40.434309 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:40.434309 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:40.434309 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:40.434309 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:40.434309 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:40.434309 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:40.434309 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:40.436145 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:40.436145 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:40.436145 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:40.436145 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:40.436145 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 19:59:40.436145 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 19:59:40.436145 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 19:59:40.436145 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 19:59:40.436145 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 19:59:40.436145 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 19:59:40.436145 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 19:59:40.436145 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 19:59:40.436145 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 334/641] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Sep 6 19:59:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:59:40.439610 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4905ec-9141-4cc7-88eb-4e90655dd197 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:40.479130 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:40.504256 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:40.545492 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 308/642] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 6 19:59:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 592 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 19:59:40.546409 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aecc23-36de-4f4f-b38d-e6dfcc2f9185 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:40.596443 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4905ec-9141-4cc7-88eb-4e90655dd197 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:40.649423 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863ba544-6d7d-424c-ad2e-e9639c1606e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:40.650420 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863ba544-6d7d-424c-ad2e-e9639c1606e7 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:40.650920 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863ba544-6d7d-424c-ad2e-e9639c1606e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:40.651385 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863ba544-6d7d-424c-ad2e-e9639c1606e7 None admin] PATH_INFO: `/v3/domains` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:40.652599 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863ba544-6d7d-424c-ad2e-e9639c1606e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:40.653147 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863ba544-6d7d-424c-ad2e-e9639c1606e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:40.655738 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863ba544-6d7d-424c-ad2e-e9639c1606e7 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:40.686037 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 335/643] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 6 19:59:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:40.687942 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863ba544-6d7d-424c-ad2e-e9639c1606e7 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:40.724892 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aecc23-36de-4f4f-b38d-e6dfcc2f9185 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:40.772137 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7407af-13b8-4711-9419-cbfd31e2956f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:40.772774 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7407af-13b8-4711-9419-cbfd31e2956f None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:40.773160 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7407af-13b8-4711-9419-cbfd31e2956f None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:40.773555 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7407af-13b8-4711-9419-cbfd31e2956f None admin] PATH_INFO: `/v3/domains` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:40.774460 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7407af-13b8-4711-9419-cbfd31e2956f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:40.774772 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7407af-13b8-4711-9419-cbfd31e2956f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:40.776321 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7407af-13b8-4711-9419-cbfd31e2956f None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:40.792149 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 309/644] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 6 19:59:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:40.793075 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7407af-13b8-4711-9419-cbfd31e2956f None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:40.847023 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:40.904306 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863ba544-6d7d-424c-ad2e-e9639c1606e7 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:40.919385 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:40.966535 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99027d5-4ac7-4a06-88b2-e46f7f5ea8a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:40.967245 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99027d5-4ac7-4a06-88b2-e46f7f5ea8a9 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:40.967538 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99027d5-4ac7-4a06-88b2-e46f7f5ea8a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:40.967817 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99027d5-4ac7-4a06-88b2-e46f7f5ea8a9 None admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:40.968522 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99027d5-4ac7-4a06-88b2-e46f7f5ea8a9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:40.968813 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99027d5-4ac7-4a06-88b2-e46f7f5ea8a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:40.970294 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99027d5-4ac7-4a06-88b2-e46f7f5ea8a9 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:40.981228 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 336/645] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Sep 6 19:59:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:40.982308 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99027d5-4ac7-4a06-88b2-e46f7f5ea8a9 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:40.983208 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7407af-13b8-4711-9419-cbfd31e2956f None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:41.039101 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce045fd-b534-4a29-bb35-7cdc95330e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:41.040355 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce045fd-b534-4a29-bb35-7cdc95330e5e None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:41.040865 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce045fd-b534-4a29-bb35-7cdc95330e5e None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:41.041353 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce045fd-b534-4a29-bb35-7cdc95330e5e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:41.063821 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce045fd-b534-4a29-bb35-7cdc95330e5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:41.066889 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce045fd-b534-4a29-bb35-7cdc95330e5e None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:41.098810 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-2ce045fd-b534-4a29-bb35-7cdc95330e5e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 19:59:41.098810 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:41.098810 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:41.098810 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:41.098810 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:41.098810 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:41.098810 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:41.098810 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:41.098810 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:41.098810 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:41.098810 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:41.098810 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:41.098810 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 19:59:41.098810 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 19:59:41.098810 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 19:59:41.098810 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 19:59:41.099878 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:41.099878 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:41.099878 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:41.099878 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:41.099878 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:41.099878 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:41.099878 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:41.099878 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:41.099878 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:41.099878 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:41.099878 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:41.099878 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:41.099878 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:41.099878 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:41.099878 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:41.099878 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:41.099878 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 19:59:41.106379 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 19:59:41.106379 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 19:59:41.106379 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 19:59:41.106379 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 310/646] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 19:59:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:59:41.106379 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce045fd-b534-4a29-bb35-7cdc95330e5e None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:41.142363 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99027d5-4ac7-4a06-88b2-e46f7f5ea8a9 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:41.190977 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a3cc0e-d838-46ac-9bf0-0a384e7bf6e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:41.191535 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a3cc0e-d838-46ac-9bf0-0a384e7bf6e6 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:41.191827 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a3cc0e-d838-46ac-9bf0-0a384e7bf6e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:41.192171 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a3cc0e-d838-46ac-9bf0-0a384e7bf6e6 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:41.201225 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a3cc0e-d838-46ac-9bf0-0a384e7bf6e6 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:41.203224 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a3cc0e-d838-46ac-9bf0-0a384e7bf6e6 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:41.217565 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-b5a3cc0e-d838-46ac-9bf0-0a384e7bf6e6 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 06 19:59:41.217565 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:41.217565 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:41.217565 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:41.217565 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:41.217565 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:41.217565 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:41.217565 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:41.217565 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:41.217565 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:41.217565 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:41.217565 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:41.217565 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 19:59:41.217565 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 19:59:41.217565 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 19:59:41.217565 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 19:59:41.218827 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:41.218827 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:41.218827 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 19:59:41.218827 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 19:59:41.218827 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 19:59:41.218827 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 19:59:41.218827 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:41.218827 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:41.218827 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:41.218827 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:41.218827 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:41.218827 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:41.218827 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:41.218827 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:41.218827 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:41.218827 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:41.218827 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:41.218827 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:41.220891 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:41.220891 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:41.220891 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 19:59:41.220891 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 19:59:41.220891 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 19:59:41.220891 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 19:59:41.220891 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 19:59:41.220891 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 19:59:41.220891 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 06 19:59:41.220891 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 19:59:41.220891 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 337/647] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 6 19:59:40 2022] GET /identity/v3/users/cinder => generated 84 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:59:41.220891 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a3cc0e-d838-46ac-9bf0-0a384e7bf6e6 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:41.252832 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:41.255774 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:41.308294 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce045fd-b534-4a29-bb35-7cdc95330e5e None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:41.353359 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a3cc0e-d838-46ac-9bf0-0a384e7bf6e6 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:41.365651 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595ee5b5-1317-4ba0-84f5-31b29dce54d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:41.366354 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595ee5b5-1317-4ba0-84f5-31b29dce54d9 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:41.366636 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595ee5b5-1317-4ba0-84f5-31b29dce54d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:41.367506 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595ee5b5-1317-4ba0-84f5-31b29dce54d9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:41.377293 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595ee5b5-1317-4ba0-84f5-31b29dce54d9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:41.378900 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595ee5b5-1317-4ba0-84f5-31b29dce54d9 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:41.392362 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-595ee5b5-1317-4ba0-84f5-31b29dce54d9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 06 19:59:41.392362 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:41.392362 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:41.392362 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:41.392362 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:41.392362 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:41.392362 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:41.392362 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:41.392362 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:41.392362 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:41.392362 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:41.392362 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:41.392362 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 19:59:41.392362 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 19:59:41.392362 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 19:59:41.392362 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 19:59:41.402681 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:41.402681 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:41.402681 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 19:59:41.402681 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 19:59:41.402681 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 19:59:41.402681 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 19:59:41.402681 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:41.402681 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:41.402681 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:41.402681 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:41.402681 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:41.402681 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:41.402681 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:41.402681 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:41.402681 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:41.402681 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:41.402681 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:41.402681 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:41.404643 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:41.404643 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:41.404643 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 19:59:41.404643 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 19:59:41.404643 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 19:59:41.404643 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 19:59:41.404643 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 19:59:41.404643 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 19:59:41.404643 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 06 19:59:41.404643 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 19:59:41.404643 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 311/648] 10.222.0.20 () {56 vars in 1093 bytes} [Tue Sep 6 19:59:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:59:41.404643 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595ee5b5-1317-4ba0-84f5-31b29dce54d9 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:41.410249 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f30846-eb57-427e-8339-175644875bed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:41.411152 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f30846-eb57-427e-8339-175644875bed None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:41.411349 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f30846-eb57-427e-8339-175644875bed None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:41.411560 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f30846-eb57-427e-8339-175644875bed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:41.425247 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f30846-eb57-427e-8339-175644875bed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:41.427620 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f30846-eb57-427e-8339-175644875bed None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:41.449812 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-86f30846-eb57-427e-8339-175644875bed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 19:59:41.449812 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:41.449812 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:41.449812 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:41.449812 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:41.449812 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:41.449812 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:41.449812 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:41.449812 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:41.449812 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:41.449812 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:41.449812 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:41.449812 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 19:59:41.449812 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 19:59:41.449812 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 19:59:41.449812 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 19:59:41.450845 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:41.450845 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:41.450845 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:41.450845 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:41.450845 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:41.450845 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:41.450845 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:41.450845 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:41.450845 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:41.450845 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:41.450845 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:41.450845 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:41.450845 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:41.450845 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:41.450845 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:41.450845 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:41.450845 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 19:59:41.451774 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 19:59:41.451774 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 19:59:41.451774 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 19:59:41.451774 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 338/649] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 19:59:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:59:41.453871 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f30846-eb57-427e-8339-175644875bed None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:41.600641 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595ee5b5-1317-4ba0-84f5-31b29dce54d9 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:41.601488 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:41.639527 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f30846-eb57-427e-8339-175644875bed None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:41.663547 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:41.673490 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f3cf17-0841-4775-8343-764f343ecdd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:41.674154 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f3cf17-0841-4775-8343-764f343ecdd1 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:41.674440 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f3cf17-0841-4775-8343-764f343ecdd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:41.674809 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f3cf17-0841-4775-8343-764f343ecdd1 None admin] PATH_INFO: `/v3/domains` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:41.675817 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f3cf17-0841-4775-8343-764f343ecdd1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:41.676139 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f3cf17-0841-4775-8343-764f343ecdd1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:41.677719 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f3cf17-0841-4775-8343-764f343ecdd1 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:41.701416 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 312/650] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 6 19:59:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:41.701416 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f3cf17-0841-4775-8343-764f343ecdd1 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:41.716193 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de925566-e161-4f43-ba02-6dccee6d7f54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:41.717542 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de925566-e161-4f43-ba02-6dccee6d7f54 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:41.718082 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de925566-e161-4f43-ba02-6dccee6d7f54 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:41.718672 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de925566-e161-4f43-ba02-6dccee6d7f54 None admin] PATH_INFO: `/v3/users` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:41.720585 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de925566-e161-4f43-ba02-6dccee6d7f54 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:41.721302 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de925566-e161-4f43-ba02-6dccee6d7f54 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:41.725203 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de925566-e161-4f43-ba02-6dccee6d7f54 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:41.760895 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 339/651] 10.222.0.20 () {56 vars in 1137 bytes} [Tue Sep 6 19:59:41 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:41.760895 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de925566-e161-4f43-ba02-6dccee6d7f54 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:41.898090 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f3cf17-0841-4775-8343-764f343ecdd1 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:41.953262 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8ef1cd-f07b-4d65-92c4-cbe18a7a53a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:41.954165 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8ef1cd-f07b-4d65-92c4-cbe18a7a53a2 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:41.954632 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8ef1cd-f07b-4d65-92c4-cbe18a7a53a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:41.954791 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de925566-e161-4f43-ba02-6dccee6d7f54 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:41.955223 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8ef1cd-f07b-4d65-92c4-cbe18a7a53a2 None admin] PATH_INFO: `/v3/users` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:41.956509 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8ef1cd-f07b-4d65-92c4-cbe18a7a53a2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:41.957131 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8ef1cd-f07b-4d65-92c4-cbe18a7a53a2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:41.960414 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8ef1cd-f07b-4d65-92c4-cbe18a7a53a2 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:41.997332 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 313/652] 10.222.0.20 () {56 vars in 1133 bytes} [Tue Sep 6 19:59:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:41.999560 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8ef1cd-f07b-4d65-92c4-cbe18a7a53a2 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:42.012100 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe0f916-aefa-41d9-ac29-3b6b039407ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:42.012956 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe0f916-aefa-41d9-ac29-3b6b039407ce None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:42.013372 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe0f916-aefa-41d9-ac29-3b6b039407ce None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:42.013831 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe0f916-aefa-41d9-ac29-3b6b039407ce None admin] PATH_INFO: `/v3/domains` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:42.014853 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe0f916-aefa-41d9-ac29-3b6b039407ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:42.015296 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe0f916-aefa-41d9-ac29-3b6b039407ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:42.017443 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe0f916-aefa-41d9-ac29-3b6b039407ce None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:42.028400 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:42.044703 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 340/653] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 6 19:59:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:42.044703 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe0f916-aefa-41d9-ac29-3b6b039407ce None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:42.103159 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:42.192429 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8ef1cd-f07b-4d65-92c4-cbe18a7a53a2 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:42.202551 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe0f916-aefa-41d9-ac29-3b6b039407ce None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:42.247900 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d6e62b-0fe3-4994-b1ab-7dc6c9a941ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:42.248547 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d6e62b-0fe3-4994-b1ab-7dc6c9a941ad None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:42.248835 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d6e62b-0fe3-4994-b1ab-7dc6c9a941ad None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:42.249116 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d6e62b-0fe3-4994-b1ab-7dc6c9a941ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:42.265427 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0925b1c0-6656-44f0-9199-9a8cbf2dae9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:42.266564 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0925b1c0-6656-44f0-9199-9a8cbf2dae9e None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:42.267037 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0925b1c0-6656-44f0-9199-9a8cbf2dae9e None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:42.267512 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0925b1c0-6656-44f0-9199-9a8cbf2dae9e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:42.281299 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0925b1c0-6656-44f0-9199-9a8cbf2dae9e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:42.284095 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0925b1c0-6656-44f0-9199-9a8cbf2dae9e None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:42.307274 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-0925b1c0-6656-44f0-9199-9a8cbf2dae9e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 19:59:42.307274 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:42.307274 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:42.307274 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:42.307274 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:42.307274 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:42.307274 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:42.307274 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:42.307274 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:42.307274 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:42.307274 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:42.307274 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:42.307274 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 19:59:42.307274 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 19:59:42.307274 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 19:59:42.308489 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 19:59:42.308489 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:42.308489 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:42.308489 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:42.308489 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:42.308489 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:42.308489 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:42.308489 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:42.308489 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:42.308489 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:42.308489 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:42.308489 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:42.308489 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:42.308489 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:42.308489 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:42.308489 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:42.308489 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:42.309739 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 19:59:42.309739 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 19:59:42.309739 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 19:59:42.309739 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 19:59:42.309739 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 19:59:42.309739 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 19:59:42.309739 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 19:59:42.309739 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 19:59:42.310188 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 314/654] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 6 19:59:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:59:42.311498 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0925b1c0-6656-44f0-9199-9a8cbf2dae9e None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:42.347672 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:42.356685 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d6e62b-0fe3-4994-b1ab-7dc6c9a941ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:42.359930 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d6e62b-0fe3-4994-b1ab-7dc6c9a941ad None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:42.453166 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0925b1c0-6656-44f0-9199-9a8cbf2dae9e None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:42.467412 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:42.501601 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cecd81-e6da-4a0d-85be-60cdab26fab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:42.502218 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cecd81-e6da-4a0d-85be-60cdab26fab9 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:42.502491 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cecd81-e6da-4a0d-85be-60cdab26fab9 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:42.502761 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cecd81-e6da-4a0d-85be-60cdab26fab9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:42.529538 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 341/655] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 6 19:59:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 19:59:42.529538 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d6e62b-0fe3-4994-b1ab-7dc6c9a941ad None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:42.620300 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cecd81-e6da-4a0d-85be-60cdab26fab9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:42.623475 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cecd81-e6da-4a0d-85be-60cdab26fab9 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:42.670658 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:42.675278 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d6e62b-0fe3-4994-b1ab-7dc6c9a941ad None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:42.719057 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b288b55f-d777-48f1-b489-0917534a3d27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:42.719749 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b288b55f-d777-48f1-b489-0917534a3d27 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:42.720030 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b288b55f-d777-48f1-b489-0917534a3d27 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:42.720304 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b288b55f-d777-48f1-b489-0917534a3d27 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:42.728881 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b288b55f-d777-48f1-b489-0917534a3d27 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:42.730425 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b288b55f-d777-48f1-b489-0917534a3d27 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:42.744185 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-b288b55f-d777-48f1-b489-0917534a3d27 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 06 19:59:42.744185 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:42.744185 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:42.744185 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:42.744185 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:42.744185 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:42.744185 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:42.744185 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:42.744185 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:42.744185 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:42.744185 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:42.744185 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:42.744185 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 19:59:42.744185 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 19:59:42.744185 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 19:59:42.744185 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 19:59:42.746105 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:42.746105 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:42.746105 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 19:59:42.746105 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 19:59:42.746105 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 19:59:42.746105 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 19:59:42.746105 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:42.746105 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:42.746105 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:42.746105 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:42.746105 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:42.746105 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:42.746105 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:42.746105 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:42.746105 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:42.746105 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:42.746105 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:42.746105 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:42.748400 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:42.748400 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:42.748400 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 19:59:42.748400 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 19:59:42.748400 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 19:59:42.748400 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 19:59:42.748400 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 19:59:42.748400 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 19:59:42.748400 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 06 19:59:42.748400 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 19:59:42.748400 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 342/656] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Sep 6 19:59:42 2022] GET /identity/v3/users/neutron => generated 85 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:59:42.748400 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b288b55f-d777-48f1-b489-0917534a3d27 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:42.767135 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 315/657] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 6 19:59:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 19:59:42.767920 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cecd81-e6da-4a0d-85be-60cdab26fab9 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:42.790627 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:42.889115 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b288b55f-d777-48f1-b489-0917534a3d27 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:42.920876 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cecd81-e6da-4a0d-85be-60cdab26fab9 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:42.953244 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a894fa29-3aff-4937-8907-5245aa23f0dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:42.954111 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a894fa29-3aff-4937-8907-5245aa23f0dd None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:42.954483 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a894fa29-3aff-4937-8907-5245aa23f0dd None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:42.954842 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a894fa29-3aff-4937-8907-5245aa23f0dd None admin] PATH_INFO: `/v3/projects` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:42.955729 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a894fa29-3aff-4937-8907-5245aa23f0dd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:42.956211 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a894fa29-3aff-4937-8907-5245aa23f0dd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:42.958118 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a894fa29-3aff-4937-8907-5245aa23f0dd None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:42.972620 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 343/658] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Sep 6 19:59:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:42.974577 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576a4add-70e4-4e45-9734-e28d7a168e8e None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:42.974861 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576a4add-70e4-4e45-9734-e28d7a168e8e None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:42.975132 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576a4add-70e4-4e45-9734-e28d7a168e8e None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:42.976505 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 344/659] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 19:59:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 19:59:42.977998 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576a4add-70e4-4e45-9734-e28d7a168e8e None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:42.990053 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c178919d-5d35-43bd-af30-e4771f2f0a80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:42.991024 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c178919d-5d35-43bd-af30-e4771f2f0a80 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:42.991392 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c178919d-5d35-43bd-af30-e4771f2f0a80 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:42.991866 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c178919d-5d35-43bd-af30-e4771f2f0a80 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:43.001532 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:43.009425 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c178919d-5d35-43bd-af30-e4771f2f0a80 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:43.011713 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c178919d-5d35-43bd-af30-e4771f2f0a80 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:43.035540 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-c178919d-5d35-43bd-af30-e4771f2f0a80 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 19:59:43.035540 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:43.035540 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:43.035540 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:43.035540 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:43.035540 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:43.035540 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:43.035540 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:43.035540 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:43.035540 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:43.035540 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:43.035540 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:43.035540 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 19:59:43.035540 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 19:59:43.035540 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 19:59:43.035540 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 19:59:43.037940 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:43.037940 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:43.037940 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:43.037940 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:43.037940 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:43.037940 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:43.037940 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:43.037940 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:43.037940 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:43.037940 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:43.037940 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:43.037940 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:43.037940 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:43.037940 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:43.037940 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:43.037940 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:43.037940 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 19:59:43.039495 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 19:59:43.039495 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 19:59:43.039495 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 19:59:43.039495 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 316/660] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 19:59:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:59:43.039495 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c178919d-5d35-43bd-af30-e4771f2f0a80 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:43.125137 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:43.135703 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576a4add-70e4-4e45-9734-e28d7a168e8e None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:43.182229 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdddfc51-70bf-4e92-8a7c-522b05834300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:43.183070 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdddfc51-70bf-4e92-8a7c-522b05834300 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:43.183459 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdddfc51-70bf-4e92-8a7c-522b05834300 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:43.183749 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdddfc51-70bf-4e92-8a7c-522b05834300 None admin] PATH_INFO: `/v3/users` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:43.184660 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdddfc51-70bf-4e92-8a7c-522b05834300 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:43.184965 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdddfc51-70bf-4e92-8a7c-522b05834300 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:43.187015 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdddfc51-70bf-4e92-8a7c-522b05834300 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:43.194004 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c178919d-5d35-43bd-af30-e4771f2f0a80 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:43.211192 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 345/661] 10.222.0.20 () {56 vars in 1139 bytes} [Tue Sep 6 19:59:42 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:43.212614 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdddfc51-70bf-4e92-8a7c-522b05834300 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:43.241916 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fca88a4-650e-4595-b9b1-cb0f736984b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:43.242567 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fca88a4-650e-4595-b9b1-cb0f736984b0 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:43.242910 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fca88a4-650e-4595-b9b1-cb0f736984b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:43.243236 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fca88a4-650e-4595-b9b1-cb0f736984b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:43.261064 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fca88a4-650e-4595-b9b1-cb0f736984b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:43.263669 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fca88a4-650e-4595-b9b1-cb0f736984b0 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:43.290172 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-8fca88a4-650e-4595-b9b1-cb0f736984b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 19:59:43.290172 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:43.290172 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:43.290172 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:43.290172 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:43.290172 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:43.290172 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:43.290172 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:43.290172 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:43.290172 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:43.290172 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:43.290172 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:43.290172 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 19:59:43.290172 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 19:59:43.290172 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 19:59:43.290172 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 19:59:43.291450 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:43.291450 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:43.291450 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:43.291450 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:43.291450 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:43.291450 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:43.291450 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:43.291450 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:43.291450 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:43.291450 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:43.291450 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:43.291450 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:43.291450 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:43.291450 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:43.291450 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:43.291450 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:43.291450 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 19:59:43.292472 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 19:59:43.292472 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 19:59:43.292472 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 19:59:43.292472 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 317/662] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 19:59:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:59:43.294074 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4120c7c9-0c5d-4120-82fc-ceb7e1226a6a None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:43.294397 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4120c7c9-0c5d-4120-82fc-ceb7e1226a6a None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:43.294712 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4120c7c9-0c5d-4120-82fc-ceb7e1226a6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:43.322675 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-4120c7c9-0c5d-4120-82fc-ceb7e1226a6a None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 19:59:43.324746 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-4120c7c9-0c5d-4120-82fc-ceb7e1226a6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 19:59:43.342673 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:43.376555 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdddfc51-70bf-4e92-8a7c-522b05834300 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:43.418565 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 318/663] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 19:59:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 19:59:43.419547 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4120c7c9-0c5d-4120-82fc-ceb7e1226a6a None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:43.431600 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f559c4-bbcf-4488-9927-8b9e2e3aa382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:43.432671 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f559c4-bbcf-4488-9927-8b9e2e3aa382 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:43.433130 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f559c4-bbcf-4488-9927-8b9e2e3aa382 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:43.433643 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f559c4-bbcf-4488-9927-8b9e2e3aa382 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:43.435078 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f559c4-bbcf-4488-9927-8b9e2e3aa382 None admin] RBAC: Adding query filter params (role.id=b950bf73c93e4f7ebed923ff14e39faa, scope.project.id=6aef43493bb1420a9e122313d8db7d5a, user.id=07af18f35ffc42a295278d5349e06381) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:43.435643 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f559c4-bbcf-4488-9927-8b9e2e3aa382 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:43.438596 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f559c4-bbcf-4488-9927-8b9e2e3aa382 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:43.455902 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 346/664] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Sep 6 19:59:43 2022] GET /identity/v3/role_assignments?user.id=07af18f35ffc42a295278d5349e06381&scope.project.id=6aef43493bb1420a9e122313d8db7d5a&role.id=b950bf73c93e4f7ebed923ff14e39faa => generated 608 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:43.458254 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f559c4-bbcf-4488-9927-8b9e2e3aa382 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:43.510099 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:43.644694 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4120c7c9-0c5d-4120-82fc-ceb7e1226a6a None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:43.681166 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f559c4-bbcf-4488-9927-8b9e2e3aa382 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:43.724407 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c116ed3-5917-466e-8e95-f8a839137443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:43.725321 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c116ed3-5917-466e-8e95-f8a839137443 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:43.726933 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c116ed3-5917-466e-8e95-f8a839137443 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:43.726933 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c116ed3-5917-466e-8e95-f8a839137443 None admin] PATH_INFO: `/v3/domains` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:43.727485 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c116ed3-5917-466e-8e95-f8a839137443 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:43.727810 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c116ed3-5917-466e-8e95-f8a839137443 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:43.730084 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c116ed3-5917-466e-8e95-f8a839137443 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:43.754872 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dadb0c-08c4-42af-8b2b-ed308d539c81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:43.755606 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dadb0c-08c4-42af-8b2b-ed308d539c81 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:43.756017 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dadb0c-08c4-42af-8b2b-ed308d539c81 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:43.756482 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dadb0c-08c4-42af-8b2b-ed308d539c81 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:43.757504 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 319/665] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 6 19:59:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:43.758432 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c116ed3-5917-466e-8e95-f8a839137443 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:43.783229 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:43.892534 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dadb0c-08c4-42af-8b2b-ed308d539c81 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:43.894768 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dadb0c-08c4-42af-8b2b-ed308d539c81 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:43.924930 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c116ed3-5917-466e-8e95-f8a839137443 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:43.930176 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:43.993922 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9ed05e-8466-4f0b-a690-b928740ad602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:43.994977 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9ed05e-8466-4f0b-a690-b928740ad602 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:43.995641 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9ed05e-8466-4f0b-a690-b928740ad602 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:43.996151 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9ed05e-8466-4f0b-a690-b928740ad602 None admin] PATH_INFO: `/v3/domains` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:43.997526 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9ed05e-8466-4f0b-a690-b928740ad602 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:43.998105 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9ed05e-8466-4f0b-a690-b928740ad602 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:44.000844 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9ed05e-8466-4f0b-a690-b928740ad602 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:44.025560 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 320/666] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 6 19:59:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:44.027356 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9ed05e-8466-4f0b-a690-b928740ad602 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:44.097894 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 347/667] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 6 19:59:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 634 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 19:59:44.097894 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dadb0c-08c4-42af-8b2b-ed308d539c81 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:44.173884 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:44.226458 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9ed05e-8466-4f0b-a690-b928740ad602 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:44.271658 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dadb0c-08c4-42af-8b2b-ed308d539c81 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:44.291810 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a3f2e4-eb7a-41c4-8de3-61be77f086d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:44.292773 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a3f2e4-eb7a-41c4-8de3-61be77f086d7 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:44.293134 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a3f2e4-eb7a-41c4-8de3-61be77f086d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:44.293504 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a3f2e4-eb7a-41c4-8de3-61be77f086d7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:44.309779 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a3f2e4-eb7a-41c4-8de3-61be77f086d7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:44.311371 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a3f2e4-eb7a-41c4-8de3-61be77f086d7 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:44.327676 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-33a3f2e4-eb7a-41c4-8de3-61be77f086d7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 19:59:44.327676 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:44.327676 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:44.327676 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:44.327676 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:44.327676 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:44.327676 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:44.327676 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:44.327676 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:44.327676 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:44.327676 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:44.327676 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:44.327676 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 19:59:44.327676 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 19:59:44.327676 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 19:59:44.329414 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 19:59:44.329414 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:44.329414 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:44.329414 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:44.329414 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:44.329414 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:44.329414 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:44.329414 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:44.329414 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:44.329414 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:44.329414 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:44.329414 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:44.329414 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:44.329414 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:44.329414 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:44.329414 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:44.329414 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:44.331788 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 19:59:44.331788 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 19:59:44.331788 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 19:59:44.331788 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 19:59:44.331788 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 19:59:44.331788 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 19:59:44.331788 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 19:59:44.331788 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 19:59:44.331788 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 321/668] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 6 19:59:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:59:44.331788 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a3f2e4-eb7a-41c4-8de3-61be77f086d7 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:44.339376 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e168cca7-5bfd-4f16-88c8-6c34a09e12e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:44.340184 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e168cca7-5bfd-4f16-88c8-6c34a09e12e1 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:44.340667 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e168cca7-5bfd-4f16-88c8-6c34a09e12e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:44.341167 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e168cca7-5bfd-4f16-88c8-6c34a09e12e1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:44.360857 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e168cca7-5bfd-4f16-88c8-6c34a09e12e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 19:59:44.365736 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:44.369823 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-e168cca7-5bfd-4f16-88c8-6c34a09e12e1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 19:59:44.369823 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:44.369823 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:44.369823 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:44.369823 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:44.369823 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:44.369823 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:44.369823 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:44.369823 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:44.369823 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:44.369823 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:44.369823 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:44.369823 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 19:59:44.369823 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 19:59:44.369823 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 19:59:44.369823 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 19:59:44.371719 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 19:59:44.371719 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 19:59:44.371719 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 19:59:44.371719 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 19:59:44.371719 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:44.371719 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:44.371719 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:44.371719 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:44.371719 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:44.371719 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:44.371719 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:44.371719 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:44.371719 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:44.371719 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:44.371719 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:44.371719 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:44.373466 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:44.373466 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:44.373466 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:44.373466 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:44.373466 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 19:59:44.373466 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 19:59:44.373466 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 19:59:44.373466 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 19:59:44.373466 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 19:59:44.373466 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 19:59:44.373466 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 19:59:44.373466 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 19:59:44.373466 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 348/669] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Sep 6 19:59:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:59:44.373466 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e168cca7-5bfd-4f16-88c8-6c34a09e12e1 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:44.498224 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a3f2e4-eb7a-41c4-8de3-61be77f086d7 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:44.575273 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a42534-a4db-481e-bb0e-a1fa36618c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:44.576298 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a42534-a4db-481e-bb0e-a1fa36618c4b None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:44.577133 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a42534-a4db-481e-bb0e-a1fa36618c4b None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:44.577296 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a42534-a4db-481e-bb0e-a1fa36618c4b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:44.579031 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:44.583236 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e168cca7-5bfd-4f16-88c8-6c34a09e12e1 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:44.590719 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a42534-a4db-481e-bb0e-a1fa36618c4b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:44.593541 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a42534-a4db-481e-bb0e-a1fa36618c4b None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:44.615813 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-84a42534-a4db-481e-bb0e-a1fa36618c4b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 19:59:44.615813 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:44.615813 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:44.615813 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:44.615813 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:44.615813 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:44.615813 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:44.615813 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:44.615813 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:44.615813 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:44.615813 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:44.615813 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:44.615813 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 19:59:44.615813 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 19:59:44.615813 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 19:59:44.617502 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 19:59:44.617502 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:44.617502 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:44.617502 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:44.617502 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:44.617502 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:44.617502 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:44.617502 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:44.617502 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:44.617502 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:44.617502 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:44.617502 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:44.617502 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:44.617502 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:44.617502 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:44.617502 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:44.617502 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:44.619331 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 19:59:44.619331 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 19:59:44.619331 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 19:59:44.619331 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 19:59:44.619331 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 19:59:44.619331 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 19:59:44.619331 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 19:59:44.619331 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 19:59:44.619331 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 322/670] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 6 19:59:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:59:44.620020 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a42534-a4db-481e-bb0e-a1fa36618c4b None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:44.674860 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab892cbe-4c6a-4288-a5dd-6553ac695cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:44.676043 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab892cbe-4c6a-4288-a5dd-6553ac695cf1 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:44.676537 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab892cbe-4c6a-4288-a5dd-6553ac695cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:44.677009 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab892cbe-4c6a-4288-a5dd-6553ac695cf1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:44.702741 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab892cbe-4c6a-4288-a5dd-6553ac695cf1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:44.704406 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab892cbe-4c6a-4288-a5dd-6553ac695cf1 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:44.727318 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-ab892cbe-4c6a-4288-a5dd-6553ac695cf1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 19:59:44.727318 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:44.727318 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:44.727318 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:44.727318 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:44.727318 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:44.727318 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:44.727318 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:44.727318 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:44.727318 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:44.727318 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:44.727318 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:44.727318 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 19:59:44.727318 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 19:59:44.727318 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 19:59:44.727318 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 19:59:44.729368 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:44.729368 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:44.729368 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:44.729368 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:44.729368 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:44.729368 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:44.729368 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:44.729368 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:44.729368 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:44.729368 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:44.729368 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:44.729368 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:44.729368 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:44.729368 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:44.729368 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:44.729368 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:44.729368 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 19:59:44.731245 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 19:59:44.731245 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 19:59:44.731245 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 19:59:44.733984 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 349/671] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 19:59:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 362 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:59:44.735474 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab892cbe-4c6a-4288-a5dd-6553ac695cf1 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:44.818014 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:44.864846 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a42534-a4db-481e-bb0e-a1fa36618c4b None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:44.974713 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf57f17-e69c-4724-b683-da5d5859d3e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:44.976972 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf57f17-e69c-4724-b683-da5d5859d3e1 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:44.978149 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf57f17-e69c-4724-b683-da5d5859d3e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:44.978645 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf57f17-e69c-4724-b683-da5d5859d3e1 None admin] PATH_INFO: `/v3/projects` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:44.980436 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf57f17-e69c-4724-b683-da5d5859d3e1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:44.981004 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf57f17-e69c-4724-b683-da5d5859d3e1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:44.984821 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf57f17-e69c-4724-b683-da5d5859d3e1 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:44.994945 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab892cbe-4c6a-4288-a5dd-6553ac695cf1 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:45.014454 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 323/672] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Sep 6 19:59:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 388 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:45.014454 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf57f17-e69c-4724-b683-da5d5859d3e1 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:45.074778 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbdf3e0-e293-4a62-ab62-bed15cbffedb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:45.075897 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbdf3e0-e293-4a62-ab62-bed15cbffedb None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:45.076391 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbdf3e0-e293-4a62-ab62-bed15cbffedb None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:45.076891 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbdf3e0-e293-4a62-ab62-bed15cbffedb None admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:45.078173 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbdf3e0-e293-4a62-ab62-bed15cbffedb None admin] RBAC: Adding query filter params (name=service) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:45.078728 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbdf3e0-e293-4a62-ab62-bed15cbffedb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:45.081496 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbdf3e0-e293-4a62-ab62-bed15cbffedb None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:45.105995 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 350/673] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Sep 6 19:59:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:45.105995 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbdf3e0-e293-4a62-ab62-bed15cbffedb None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:45.129859 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:45.218250 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf57f17-e69c-4724-b683-da5d5859d3e1 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:45.298392 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2067afed-6735-4fc7-bf50-ad5de23561a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:45.299176 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2067afed-6735-4fc7-bf50-ad5de23561a5 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:45.299634 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2067afed-6735-4fc7-bf50-ad5de23561a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:45.305477 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2067afed-6735-4fc7-bf50-ad5de23561a5 None admin] PATH_INFO: `/v3/projects` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:45.305477 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2067afed-6735-4fc7-bf50-ad5de23561a5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:45.305477 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2067afed-6735-4fc7-bf50-ad5de23561a5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:45.305477 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2067afed-6735-4fc7-bf50-ad5de23561a5 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:45.308693 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbdf3e0-e293-4a62-ab62-bed15cbffedb None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:45.332459 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 324/674] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Sep 6 19:59:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:45.332459 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2067afed-6735-4fc7-bf50-ad5de23561a5 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:45.337456 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:45.381127 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5958f4a3-0497-4353-9672-399596ef8e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:45.382039 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5958f4a3-0497-4353-9672-399596ef8e10 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:45.382505 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5958f4a3-0497-4353-9672-399596ef8e10 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:45.382968 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5958f4a3-0497-4353-9672-399596ef8e10 None admin] PATH_INFO: `/v3/domains` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:45.383910 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5958f4a3-0497-4353-9672-399596ef8e10 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:45.384402 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5958f4a3-0497-4353-9672-399596ef8e10 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:45.386273 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5958f4a3-0497-4353-9672-399596ef8e10 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:45.409096 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 351/675] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 6 19:59:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:45.409096 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5958f4a3-0497-4353-9672-399596ef8e10 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:45.527815 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2067afed-6735-4fc7-bf50-ad5de23561a5 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:45.551497 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:45.583660 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a01bc7-56b2-45de-b6e9-25b2d94625a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:45.584395 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a01bc7-56b2-45de-b6e9-25b2d94625a9 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:45.584812 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a01bc7-56b2-45de-b6e9-25b2d94625a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:45.585220 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a01bc7-56b2-45de-b6e9-25b2d94625a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:45.586377 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a01bc7-56b2-45de-b6e9-25b2d94625a9 None admin] RBAC: Adding query filter params (role.id=b950bf73c93e4f7ebed923ff14e39faa, scope.project.id=6aef43493bb1420a9e122313d8db7d5a, user.id=df1bca993bc142908cc7cf85eabd2b24) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:45.586827 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a01bc7-56b2-45de-b6e9-25b2d94625a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:45.588849 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a01bc7-56b2-45de-b6e9-25b2d94625a9 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:45.597715 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 325/676] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Sep 6 19:59:45 2022] GET /identity/v3/role_assignments?user.id=df1bca993bc142908cc7cf85eabd2b24&scope.project.id=6aef43493bb1420a9e122313d8db7d5a&role.id=b950bf73c93e4f7ebed923ff14e39faa => generated 608 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:45.598911 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a01bc7-56b2-45de-b6e9-25b2d94625a9 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:45.610398 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5958f4a3-0497-4353-9672-399596ef8e10 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:45.713454 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c844a050-2d5d-4314-92b0-f271f411bf90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:45.715051 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c844a050-2d5d-4314-92b0-f271f411bf90 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:45.715820 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c844a050-2d5d-4314-92b0-f271f411bf90 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:45.716574 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c844a050-2d5d-4314-92b0-f271f411bf90 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:45.748152 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:45.753522 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c844a050-2d5d-4314-92b0-f271f411bf90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:45.757716 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c844a050-2d5d-4314-92b0-f271f411bf90 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:45.788720 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-c844a050-2d5d-4314-92b0-f271f411bf90 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 19:59:45.788720 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:45.788720 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:45.788720 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:45.788720 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:45.788720 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:45.788720 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:45.788720 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:45.788720 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:45.788720 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:45.788720 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:45.788720 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:45.788720 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 19:59:45.788720 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 19:59:45.788720 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 19:59:45.788720 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 19:59:45.790486 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:45.790486 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:45.790486 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:45.790486 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:45.790486 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:45.790486 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:45.790486 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:45.790486 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:45.790486 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:45.790486 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:45.790486 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:45.790486 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:45.790486 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:45.790486 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:45.790486 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:45.790486 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:45.790486 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 19:59:45.792184 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 19:59:45.792184 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 19:59:45.792184 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 19:59:45.793848 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 352/677] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 19:59:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 389 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:59:45.795628 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c844a050-2d5d-4314-92b0-f271f411bf90 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:45.817353 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a01bc7-56b2-45de-b6e9-25b2d94625a9 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:45.873740 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffdc4bf-5bc0-474c-affe-3b6c82364a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:45.874878 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffdc4bf-5bc0-474c-affe-3b6c82364a40 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:45.875324 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffdc4bf-5bc0-474c-affe-3b6c82364a40 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:45.875790 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffdc4bf-5bc0-474c-affe-3b6c82364a40 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:45.876758 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffdc4bf-5bc0-474c-affe-3b6c82364a40 None admin] RBAC: Adding query filter params (role.id=b950bf73c93e4f7ebed923ff14e39faa, scope.project.id=6aef43493bb1420a9e122313d8db7d5a, user.id=5776cbf4f6714b2083c3ebdd6d98cc87) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:45.877238 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffdc4bf-5bc0-474c-affe-3b6c82364a40 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:45.879333 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffdc4bf-5bc0-474c-affe-3b6c82364a40 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:45.894124 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 326/678] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Sep 6 19:59:45 2022] GET /identity/v3/role_assignments?user.id=5776cbf4f6714b2083c3ebdd6d98cc87&scope.project.id=6aef43493bb1420a9e122313d8db7d5a&role.id=b950bf73c93e4f7ebed923ff14e39faa => generated 608 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:45.896065 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffdc4bf-5bc0-474c-affe-3b6c82364a40 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:46.019127 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c844a050-2d5d-4314-92b0-f271f411bf90 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:46.072177 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:46.074940 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffdc4bf-5bc0-474c-affe-3b6c82364a40 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:46.090388 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103d1f9d-6141-4aad-8893-c9d597c43aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:46.091764 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103d1f9d-6141-4aad-8893-c9d597c43aef None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:46.092425 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103d1f9d-6141-4aad-8893-c9d597c43aef None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:46.093047 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103d1f9d-6141-4aad-8893-c9d597c43aef None admin] PATH_INFO: `/v3/projects/service` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:46.109895 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103d1f9d-6141-4aad-8893-c9d597c43aef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:46.112372 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103d1f9d-6141-4aad-8893-c9d597c43aef None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:46.135069 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cdd6c9-e6fa-4c55-bdd3-0b1947fca5b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:46.136566 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cdd6c9-e6fa-4c55-bdd3-0b1947fca5b6 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:46.137197 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cdd6c9-e6fa-4c55-bdd3-0b1947fca5b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:46.137996 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cdd6c9-e6fa-4c55-bdd3-0b1947fca5b6 None admin] PATH_INFO: `/v3/domains` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:46.139109 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cdd6c9-e6fa-4c55-bdd3-0b1947fca5b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:46.139678 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cdd6c9-e6fa-4c55-bdd3-0b1947fca5b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:46.139826 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-103d1f9d-6141-4aad-8893-c9d597c43aef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 19:59:46.139826 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:46.139826 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:46.139826 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:46.139826 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:46.139826 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:46.139826 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:46.139826 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:46.139826 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:46.139826 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:46.139826 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:46.139826 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:46.139826 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 19:59:46.139826 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 19:59:46.139826 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 19:59:46.140675 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 19:59:46.140675 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:46.140675 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:46.140675 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:46.140675 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:46.140675 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:46.140675 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:46.140675 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:46.140675 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:46.140675 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:46.140675 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:46.140675 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:46.140675 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:46.140675 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:46.140675 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:46.140675 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:46.140675 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:46.141906 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 19:59:46.141906 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 19:59:46.141906 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 19:59:46.141906 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 19:59:46.141906 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 19:59:46.141906 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 19:59:46.141906 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 19:59:46.141906 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 19:59:46.142745 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 353/679] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 6 19:59:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:59:46.144128 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:46.151718 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cdd6c9-e6fa-4c55-bdd3-0b1947fca5b6 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:46.154481 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103d1f9d-6141-4aad-8893-c9d597c43aef None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:46.179323 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 327/680] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 6 19:59:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:46.185935 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cdd6c9-e6fa-4c55-bdd3-0b1947fca5b6 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:46.389703 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103d1f9d-6141-4aad-8893-c9d597c43aef None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:46.412328 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cdd6c9-e6fa-4c55-bdd3-0b1947fca5b6 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:46.471882 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c93c93-712a-4cf6-89b6-eab9e33dc5c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:46.473311 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c93c93-712a-4cf6-89b6-eab9e33dc5c4 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:46.474055 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c93c93-712a-4cf6-89b6-eab9e33dc5c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:46.474761 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c93c93-712a-4cf6-89b6-eab9e33dc5c4 None admin] PATH_INFO: `/v3/projects` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:46.476654 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c93c93-712a-4cf6-89b6-eab9e33dc5c4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:46.477596 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c93c93-712a-4cf6-89b6-eab9e33dc5c4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:46.480573 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c93c93-712a-4cf6-89b6-eab9e33dc5c4 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:46.496068 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a26bbec-cbc9-4b02-9254-3f47a13790f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:46.497492 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a26bbec-cbc9-4b02-9254-3f47a13790f7 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:46.498152 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a26bbec-cbc9-4b02-9254-3f47a13790f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:46.499565 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a26bbec-cbc9-4b02-9254-3f47a13790f7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:46.513765 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a26bbec-cbc9-4b02-9254-3f47a13790f7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:46.514057 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 354/681] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Sep 6 19:59:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:46.516930 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a26bbec-cbc9-4b02-9254-3f47a13790f7 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:46.522828 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c93c93-712a-4cf6-89b6-eab9e33dc5c4 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:46.554494 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-3a26bbec-cbc9-4b02-9254-3f47a13790f7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 06 19:59:46.554494 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:46.554494 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:46.554494 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:46.554494 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:46.554494 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:46.554494 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:46.554494 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:46.554494 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:46.554494 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:46.554494 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:46.554494 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:46.554494 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 19:59:46.554494 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 19:59:46.554494 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 19:59:46.554494 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 19:59:46.556597 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:46.556597 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:46.556597 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 19:59:46.556597 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 19:59:46.556597 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 19:59:46.556597 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 19:59:46.556597 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:46.556597 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:46.556597 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:46.556597 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:46.556597 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:46.556597 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:46.556597 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:46.556597 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:46.556597 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:46.556597 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:46.556597 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:46.556597 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:46.558769 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:46.558769 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:46.558769 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 19:59:46.558769 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 19:59:46.558769 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 19:59:46.558769 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 19:59:46.558769 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 19:59:46.558769 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 19:59:46.558769 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 06 19:59:46.558769 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 19:59:46.563329 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 328/682] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Sep 6 19:59:46 2022] GET /identity/v3/users/swift => generated 83 bytes in 378 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:59:46.571373 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a26bbec-cbc9-4b02-9254-3f47a13790f7 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:46.651991 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:46.668363 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:46.751967 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a26bbec-cbc9-4b02-9254-3f47a13790f7 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:46.775412 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c93c93-712a-4cf6-89b6-eab9e33dc5c4 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:46.813109 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa4937f-83ea-4609-88ca-04870fb54305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:46.814052 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa4937f-83ea-4609-88ca-04870fb54305 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:46.814483 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa4937f-83ea-4609-88ca-04870fb54305 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:46.814927 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa4937f-83ea-4609-88ca-04870fb54305 None admin] PATH_INFO: `/v3/users` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:46.816637 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa4937f-83ea-4609-88ca-04870fb54305 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:46.817162 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa4937f-83ea-4609-88ca-04870fb54305 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:46.819671 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa4937f-83ea-4609-88ca-04870fb54305 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:46.850822 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 329/683] 10.222.0.20 () {56 vars in 1135 bytes} [Tue Sep 6 19:59:46 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:46.852557 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fd4d16-a099-4052-876a-01281904a5ae None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:46.852965 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fd4d16-a099-4052-876a-01281904a5ae None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:46.853410 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fd4d16-a099-4052-876a-01281904a5ae None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:46.855972 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 330/684] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 19:59:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 19:59:46.867215 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fd4d16-a099-4052-876a-01281904a5ae None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:46.867996 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15f0282-996e-417a-8afc-66a390a364d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:46.869043 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15f0282-996e-417a-8afc-66a390a364d4 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:46.869598 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15f0282-996e-417a-8afc-66a390a364d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:46.870114 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15f0282-996e-417a-8afc-66a390a364d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:46.871542 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15f0282-996e-417a-8afc-66a390a364d4 None admin] RBAC: Adding query filter params (role.id=b950bf73c93e4f7ebed923ff14e39faa, scope.project.id=6aef43493bb1420a9e122313d8db7d5a, user.id=32882d38295046fea004a659a71bde58) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:46.872079 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15f0282-996e-417a-8afc-66a390a364d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:46.874878 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15f0282-996e-417a-8afc-66a390a364d4 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:46.887935 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 355/685] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Sep 6 19:59:46 2022] GET /identity/v3/role_assignments?user.id=32882d38295046fea004a659a71bde58&scope.project.id=6aef43493bb1420a9e122313d8db7d5a&role.id=b950bf73c93e4f7ebed923ff14e39faa => generated 608 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 19:59:46.890428 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5727b35a-4fd4-4a4d-b749-150350d0d40c None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:46.890910 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5727b35a-4fd4-4a4d-b749-150350d0d40c None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:46.891373 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5727b35a-4fd4-4a4d-b749-150350d0d40c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:46.939548 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-5727b35a-4fd4-4a4d-b749-150350d0d40c None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 19:59:46.944482 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-5727b35a-4fd4-4a4d-b749-150350d0d40c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 19:59:47.126849 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fd4d16-a099-4052-876a-01281904a5ae None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:47.163897 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:47.173415 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 356/686] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 19:59:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 19:59:47.231079 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915784c1-2544-49f6-b145-3f485c141096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:47.231991 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915784c1-2544-49f6-b145-3f485c141096 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:47.232479 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915784c1-2544-49f6-b145-3f485c141096 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:47.232963 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915784c1-2544-49f6-b145-3f485c141096 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:47.346003 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915784c1-2544-49f6-b145-3f485c141096 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:47.347773 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915784c1-2544-49f6-b145-3f485c141096 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:47.485727 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 331/687] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 6 19:59:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 619 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 19:59:47.492619 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5727b35a-4fd4-4a4d-b749-150350d0d40c None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:47.572457 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915784c1-2544-49f6-b145-3f485c141096 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:47.587378 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:47.599382 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:47.766668 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915784c1-2544-49f6-b145-3f485c141096 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:47.782310 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5727b35a-4fd4-4a4d-b749-150350d0d40c None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:47.832251 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea822a2c-bccc-44bf-8a45-9eb586e1a3d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:47.832922 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea822a2c-bccc-44bf-8a45-9eb586e1a3d7 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:47.833206 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea822a2c-bccc-44bf-8a45-9eb586e1a3d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:47.833524 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea822a2c-bccc-44bf-8a45-9eb586e1a3d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:47.849798 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea822a2c-bccc-44bf-8a45-9eb586e1a3d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:47.852600 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea822a2c-bccc-44bf-8a45-9eb586e1a3d7 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:47.876089 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-ea822a2c-bccc-44bf-8a45-9eb586e1a3d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 19:59:47.876089 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:47.876089 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:47.876089 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:47.876089 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:47.876089 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:47.876089 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:47.876089 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:47.876089 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:47.876089 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:47.876089 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:47.876089 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:47.876089 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 19:59:47.876089 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 19:59:47.876089 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 19:59:47.876089 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 19:59:47.877925 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:47.877925 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:47.877925 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:47.877925 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:47.877925 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:47.877925 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:47.877925 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:47.877925 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:47.877925 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:47.877925 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:47.877925 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:47.877925 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:47.877925 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:47.877925 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:47.877925 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:47.877925 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:47.877925 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 19:59:47.879746 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 19:59:47.879746 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 19:59:47.879746 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 19:59:47.879746 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 332/688] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 19:59:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 19:59:47.893118 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea822a2c-bccc-44bf-8a45-9eb586e1a3d7 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:47.935896 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896a23c6-07b7-421f-b531-2bde37996dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:47.937009 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896a23c6-07b7-421f-b531-2bde37996dcf None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:47.937509 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896a23c6-07b7-421f-b531-2bde37996dcf None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:47.938029 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896a23c6-07b7-421f-b531-2bde37996dcf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:47.985566 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896a23c6-07b7-421f-b531-2bde37996dcf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:47.989188 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896a23c6-07b7-421f-b531-2bde37996dcf None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:48.043313 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-896a23c6-07b7-421f-b531-2bde37996dcf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 19:59:48.043313 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:48.043313 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:48.043313 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:48.043313 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:48.043313 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:48.043313 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:48.043313 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:48.043313 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:48.043313 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:48.043313 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:48.043313 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:48.043313 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 19:59:48.043313 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 19:59:48.043313 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 19:59:48.043313 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 19:59:48.046915 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:48.046915 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:48.046915 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:48.046915 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:48.046915 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:48.046915 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:48.046915 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:48.046915 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:48.046915 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:48.046915 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:48.046915 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:48.046915 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:48.046915 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:48.046915 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:48.046915 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:48.046915 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:48.046915 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 19:59:48.050986 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 19:59:48.050986 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 19:59:48.050986 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 19:59:48.050986 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 357/689] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 19:59:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 558 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:59:48.064524 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896a23c6-07b7-421f-b531-2bde37996dcf None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:48.146341 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:48.239811 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea822a2c-bccc-44bf-8a45-9eb586e1a3d7 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:48.331131 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2866d43-8ca3-488e-809d-ea117924618e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:48.332090 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2866d43-8ca3-488e-809d-ea117924618e None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:48.332585 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2866d43-8ca3-488e-809d-ea117924618e None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:48.333083 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2866d43-8ca3-488e-809d-ea117924618e None admin] PATH_INFO: `/v3/domains` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:48.334350 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2866d43-8ca3-488e-809d-ea117924618e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:48.334916 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2866d43-8ca3-488e-809d-ea117924618e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:48.337686 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2866d43-8ca3-488e-809d-ea117924618e None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:48.387946 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 333/690] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 6 19:59:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 496 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 19:59:48.399451 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2866d43-8ca3-488e-809d-ea117924618e None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:48.469031 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:48.500013 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896a23c6-07b7-421f-b531-2bde37996dcf None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:48.661566 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80be414b-3836-434d-a3e8-39fb11c0fd2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:48.663286 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80be414b-3836-434d-a3e8-39fb11c0fd2a None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:48.664884 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80be414b-3836-434d-a3e8-39fb11c0fd2a None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:48.665771 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80be414b-3836-434d-a3e8-39fb11c0fd2a None admin] PATH_INFO: `/v3/domains` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:48.667379 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80be414b-3836-434d-a3e8-39fb11c0fd2a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:48.668274 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80be414b-3836-434d-a3e8-39fb11c0fd2a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:48.671897 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80be414b-3836-434d-a3e8-39fb11c0fd2a None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:48.713840 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 358/691] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 6 19:59:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 652 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 19:59:48.722446 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80be414b-3836-434d-a3e8-39fb11c0fd2a None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:48.757482 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2866d43-8ca3-488e-809d-ea117924618e None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:48.819428 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:48.850566 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219f4ca8-09c2-4d2c-99a8-f5ce191638f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:48.851634 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219f4ca8-09c2-4d2c-99a8-f5ce191638f3 None admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:48.852209 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219f4ca8-09c2-4d2c-99a8-f5ce191638f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:48.852702 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219f4ca8-09c2-4d2c-99a8-f5ce191638f3 None admin] PATH_INFO: `/v3/users` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:48.854419 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219f4ca8-09c2-4d2c-99a8-f5ce191638f3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:48.861521 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219f4ca8-09c2-4d2c-99a8-f5ce191638f3 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:48.936246 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 334/692] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Sep 6 19:59:48 2022] POST /identity/v3/users => generated 254 bytes in 538 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 19:59:49.083053 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80be414b-3836-434d-a3e8-39fb11c0fd2a None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:49.187315 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f756db3-460b-4061-9049-d92dae70ccb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:49.188327 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f756db3-460b-4061-9049-d92dae70ccb3 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:49.188821 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f756db3-460b-4061-9049-d92dae70ccb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:49.189291 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f756db3-460b-4061-9049-d92dae70ccb3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:49.204758 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f756db3-460b-4061-9049-d92dae70ccb3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:49.207278 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f756db3-460b-4061-9049-d92dae70ccb3 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:49.233930 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-3f756db3-460b-4061-9049-d92dae70ccb3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 19:59:49.233930 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:49.233930 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:49.233930 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:49.233930 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:49.233930 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:49.233930 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:49.233930 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:49.233930 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:49.233930 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:49.233930 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:49.233930 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:49.233930 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 19:59:49.233930 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 19:59:49.233930 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 19:59:49.235045 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 19:59:49.235045 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:49.235045 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:49.235045 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:49.235045 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:49.235045 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:49.235045 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:49.235045 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:49.235045 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:49.235045 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:49.235045 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:49.235045 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:49.235045 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:49.235045 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:49.235045 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:49.235045 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:49.235045 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:49.236010 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 19:59:49.236010 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 19:59:49.236010 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 19:59:49.236010 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 19:59:49.236010 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 19:59:49.236010 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 19:59:49.236010 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 19:59:49.236010 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 19:59:49.237419 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 359/693] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 6 19:59:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 515 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 19:59:49.243555 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219f4ca8-09c2-4d2c-99a8-f5ce191638f3 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:49.452921 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219f4ca8-09c2-4d2c-99a8-f5ce191638f3 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:49.524194 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a018a27-ba76-4b11-b893-6e9ed64dc526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:49.525396 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a018a27-ba76-4b11-b893-6e9ed64dc526 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:49.525993 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a018a27-ba76-4b11-b893-6e9ed64dc526 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:49.526542 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a018a27-ba76-4b11-b893-6e9ed64dc526 None admin] PATH_INFO: `/v3/projects` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:49.527856 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a018a27-ba76-4b11-b893-6e9ed64dc526 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:49.528463 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a018a27-ba76-4b11-b893-6e9ed64dc526 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:49.531241 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a018a27-ba76-4b11-b893-6e9ed64dc526 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:49.565385 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 335/694] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Sep 6 19:59:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:49.574740 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f756db3-460b-4061-9049-d92dae70ccb3 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:49.618404 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:49.803627 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f756db3-460b-4061-9049-d92dae70ccb3 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:49.870805 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25c6ef8-0c9c-4350-ae1c-87b6dca49aa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:49.871938 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25c6ef8-0c9c-4350-ae1c-87b6dca49aa9 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:49.872433 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25c6ef8-0c9c-4350-ae1c-87b6dca49aa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:49.872915 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25c6ef8-0c9c-4350-ae1c-87b6dca49aa9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:49.874379 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25c6ef8-0c9c-4350-ae1c-87b6dca49aa9 None admin] RBAC: Adding query filter params (role.id=b950bf73c93e4f7ebed923ff14e39faa, scope.project.id=6aef43493bb1420a9e122313d8db7d5a, user.id=2b72516236da47e292d78de201b0552b) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:49.874934 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25c6ef8-0c9c-4350-ae1c-87b6dca49aa9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:49.878464 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25c6ef8-0c9c-4350-ae1c-87b6dca49aa9 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:49.896584 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 360/695] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Sep 6 19:59:49 2022] GET /identity/v3/role_assignments?user.id=2b72516236da47e292d78de201b0552b&scope.project.id=6aef43493bb1420a9e122313d8db7d5a&role.id=b950bf73c93e4f7ebed923ff14e39faa => generated 608 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 19:59:50.006792 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87f844f-5223-4fc0-b530-e152d92e600f None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:50.007279 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87f844f-5223-4fc0-b530-e152d92e600f None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:50.007765 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87f844f-5223-4fc0-b530-e152d92e600f None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:50.011001 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 336/696] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 19:59:50 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 19:59:50.021987 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b53afa6-41e8-47fc-9b4f-7a9869d25691 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:50.022497 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b53afa6-41e8-47fc-9b4f-7a9869d25691 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:50.022999 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b53afa6-41e8-47fc-9b4f-7a9869d25691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:50.058985 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-2b53afa6-41e8-47fc-9b4f-7a9869d25691 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 19:59:50.060311 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-2b53afa6-41e8-47fc-9b4f-7a9869d25691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 19:59:50.193486 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 361/697] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 19:59:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 19:59:50.430395 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94646149-9983-4795-9835-d8a904785928 None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:50.431036 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94646149-9983-4795-9835-d8a904785928 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:50.431410 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94646149-9983-4795-9835-d8a904785928 None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:50.434746 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 337/698] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 19:59:50 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 19:59:50.442223 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b5ffe4-fee8-40df-a87a-7a9efdd22086 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:50.442923 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b5ffe4-fee8-40df-a87a-7a9efdd22086 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:50.443418 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b5ffe4-fee8-40df-a87a-7a9efdd22086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:50.461397 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:50.474244 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-a9b5ffe4-fee8-40df-a87a-7a9efdd22086 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 19:59:50.475904 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-a9b5ffe4-fee8-40df-a87a-7a9efdd22086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 19:59:50.588978 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 362/699] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 19:59:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 19:59:50.681049 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94646149-9983-4795-9835-d8a904785928 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:50.736994 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:50.833008 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94646149-9983-4795-9835-d8a904785928 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:50.881317 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b7f3bf-f20e-44a1-8fdf-0fdf45122ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:50.882020 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b7f3bf-f20e-44a1-8fdf-0fdf45122ff3 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:50.882301 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b7f3bf-f20e-44a1-8fdf-0fdf45122ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:50.882706 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b7f3bf-f20e-44a1-8fdf-0fdf45122ff3 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:50.883331 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b7f3bf-f20e-44a1-8fdf-0fdf45122ff3 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:50.884854 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b7f3bf-f20e-44a1-8fdf-0fdf45122ff3 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:50.902703 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-b3b7f3bf-f20e-44a1-8fdf-0fdf45122ff3 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 06 19:59:50.902703 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:50.902703 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 19:59:50.902703 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 19:59:50.902703 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 19:59:50.902703 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 19:59:50.902703 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 19:59:50.902703 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 19:59:50.902703 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 06 19:59:50.902703 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 19:59:50.902703 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 19:59:50.902703 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 19:59:50.902703 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:50.902703 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:50.902703 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:50.902703 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:50.902703 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:50.903857 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:50.903857 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:50.903857 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:50.903857 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:50.903857 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:50.903857 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:50.903857 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 19:59:50.903857 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 19:59:50.903857 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 19:59:50.903857 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 19:59:50.903857 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:50.903857 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:50.903857 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:50.903857 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:50.903857 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:50.903857 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:50.905977 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:50.905977 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:50.905977 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:50.905977 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:50.905977 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:50.905977 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:50.905977 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:50.905977 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:50.905977 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:50.905977 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:50.905977 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 19:59:50.905977 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 19:59:50.905977 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 06 19:59:50.905977 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 19:59:50.905977 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 338/700] 10.222.0.20 () {56 vars in 1129 bytes} [Tue Sep 6 19:59:50 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 19:59:50.915639 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b5ffe4-fee8-40df-a87a-7a9efdd22086 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:50.990136 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b7f3bf-f20e-44a1-8fdf-0fdf45122ff3 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:51.136098 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b5ffe4-fee8-40df-a87a-7a9efdd22086 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:51.172694 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b7f3bf-f20e-44a1-8fdf-0fdf45122ff3 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:51.176190 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:51.269330 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d68c686-a4a7-4632-84f8-182351aead17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:51.270480 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d68c686-a4a7-4632-84f8-182351aead17 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:51.270954 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d68c686-a4a7-4632-84f8-182351aead17 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:51.271535 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d68c686-a4a7-4632-84f8-182351aead17 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:51.282864 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623ff082-669a-40d6-9132-0383ebaa058c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:51.284452 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623ff082-669a-40d6-9132-0383ebaa058c None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:51.284943 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623ff082-669a-40d6-9132-0383ebaa058c None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:51.285440 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623ff082-669a-40d6-9132-0383ebaa058c None admin] PATH_INFO: `/v3/services` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:51.286863 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623ff082-669a-40d6-9132-0383ebaa058c None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:51.287250 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623ff082-669a-40d6-9132-0383ebaa058c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:51.289995 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623ff082-669a-40d6-9132-0383ebaa058c None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:51.294924 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d68c686-a4a7-4632-84f8-182351aead17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 19:59:51.306106 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:51.308325 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-7d68c686-a4a7-4632-84f8-182351aead17 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 19:59:51.308325 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:51.308325 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:51.308325 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:51.308325 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:51.308325 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:51.308325 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:51.308325 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:51.308325 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:51.308325 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:51.308325 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:51.308325 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:51.308325 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 19:59:51.308325 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 19:59:51.308325 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 19:59:51.308325 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 19:59:51.310643 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 19:59:51.310643 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 19:59:51.310643 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 19:59:51.310643 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 19:59:51.310643 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:51.310643 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:51.310643 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:51.310643 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:51.310643 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:51.310643 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:51.310643 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:51.310643 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:51.310643 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:51.310643 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:51.310643 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:51.310643 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:51.315455 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:51.315455 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:51.315455 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:51.315455 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:51.315455 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 19:59:51.315455 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 19:59:51.315455 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 19:59:51.315455 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 19:59:51.315455 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 19:59:51.315455 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 19:59:51.315455 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 19:59:51.315455 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 19:59:51.319558 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 363/701] 10.222.0.20 () {56 vars in 1124 bytes} [Tue Sep 6 19:59:50 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 394 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:51.319821 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 339/702] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Sep 6 19:59:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:59:51.326839 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623ff082-669a-40d6-9132-0383ebaa058c None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:51.328824 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d68c686-a4a7-4632-84f8-182351aead17 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:51.539044 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d68c686-a4a7-4632-84f8-182351aead17 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:51.611002 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd68a3b-b3da-4b5d-a5eb-d9892b2e3d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:51.612648 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd68a3b-b3da-4b5d-a5eb-d9892b2e3d12 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:51.613152 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd68a3b-b3da-4b5d-a5eb-d9892b2e3d12 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:51.613670 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd68a3b-b3da-4b5d-a5eb-d9892b2e3d12 None admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:51.615022 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd68a3b-b3da-4b5d-a5eb-d9892b2e3d12 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:51.615659 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd68a3b-b3da-4b5d-a5eb-d9892b2e3d12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:51.618722 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd68a3b-b3da-4b5d-a5eb-d9892b2e3d12 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:51.636552 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 340/703] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 6 19:59:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:51.655299 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd68a3b-b3da-4b5d-a5eb-d9892b2e3d12 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:51.670836 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623ff082-669a-40d6-9132-0383ebaa058c None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:51.704743 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:51.748880 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c864dd4-2a80-4d88-a5b9-86bac39335fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:51.750297 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c864dd4-2a80-4d88-a5b9-86bac39335fe None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:51.750961 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c864dd4-2a80-4d88-a5b9-86bac39335fe None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:51.751646 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c864dd4-2a80-4d88-a5b9-86bac39335fe None admin] PATH_INFO: `/v3/services` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:51.753369 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c864dd4-2a80-4d88-a5b9-86bac39335fe None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:51.754608 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c864dd4-2a80-4d88-a5b9-86bac39335fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:51.757722 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c864dd4-2a80-4d88-a5b9-86bac39335fe None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:51.770336 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 364/704] 10.222.0.20 () {56 vars in 1124 bytes} [Tue Sep 6 19:59:51 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 444 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:51.773347 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b98a7c4-cae3-4078-a472-44708330c7d8 None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:51.774537 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b98a7c4-cae3-4078-a472-44708330c7d8 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:51.775485 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b98a7c4-cae3-4078-a472-44708330c7d8 None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:51.784928 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 365/705] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 19:59:51 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 19:59:51.789213 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac6d764-3efc-4b05-8bc4-2b7a9524cdb4 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:51.790107 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac6d764-3efc-4b05-8bc4-2b7a9524cdb4 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:51.790837 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac6d764-3efc-4b05-8bc4-2b7a9524cdb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:51.826920 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-5ac6d764-3efc-4b05-8bc4-2b7a9524cdb4 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 19:59:51.829697 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-5ac6d764-3efc-4b05-8bc4-2b7a9524cdb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 19:59:51.881710 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd68a3b-b3da-4b5d-a5eb-d9892b2e3d12 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:51.948789 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37a30ca-d4aa-42f2-b4cb-ca329e9f83c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:51.949966 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37a30ca-d4aa-42f2-b4cb-ca329e9f83c6 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:51.950448 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37a30ca-d4aa-42f2-b4cb-ca329e9f83c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:51.951022 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37a30ca-d4aa-42f2-b4cb-ca329e9f83c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:51.973717 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37a30ca-d4aa-42f2-b4cb-ca329e9f83c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:51.977103 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37a30ca-d4aa-42f2-b4cb-ca329e9f83c6 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:52.004600 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-b37a30ca-d4aa-42f2-b4cb-ca329e9f83c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 19:59:52.004600 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:52.004600 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:52.004600 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:52.004600 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:52.004600 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:52.004600 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:52.004600 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:52.004600 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:52.004600 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:52.004600 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:52.004600 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:52.004600 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 19:59:52.004600 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 19:59:52.004600 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 19:59:52.004600 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 19:59:52.006942 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:52.006942 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:52.006942 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:52.006942 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:52.006942 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:52.006942 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:52.006942 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:52.006942 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:52.006942 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:52.006942 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:52.006942 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:52.006942 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:52.006942 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:52.006942 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:52.006942 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:52.006942 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:52.006942 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 19:59:52.009187 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 19:59:52.009187 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 19:59:52.009187 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 19:59:52.010762 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 341/706] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 19:59:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 356 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:59:52.019262 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:52.022732 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37a30ca-d4aa-42f2-b4cb-ca329e9f83c6 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:52.062573 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 366/707] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 19:59:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 19:59:52.193192 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:52.238534 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37a30ca-d4aa-42f2-b4cb-ca329e9f83c6 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:52.305834 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92df328-7fc1-486f-aa6a-c539fb9421ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:52.306851 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92df328-7fc1-486f-aa6a-c539fb9421ef None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:52.307342 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92df328-7fc1-486f-aa6a-c539fb9421ef None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:52.307829 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92df328-7fc1-486f-aa6a-c539fb9421ef None admin] PATH_INFO: `/v3/domains` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:52.309031 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92df328-7fc1-486f-aa6a-c539fb9421ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:52.309572 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92df328-7fc1-486f-aa6a-c539fb9421ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:52.312161 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92df328-7fc1-486f-aa6a-c539fb9421ef None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:52.336945 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 342/708] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 6 19:59:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 19:59:52.344288 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac6d764-3efc-4b05-8bc4-2b7a9524cdb4 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:52.575782 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac6d764-3efc-4b05-8bc4-2b7a9524cdb4 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:52.660642 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5582f8b7-4b0d-4bef-9ce6-41cecadf4a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:52.661659 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5582f8b7-4b0d-4bef-9ce6-41cecadf4a4c None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:52.662134 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5582f8b7-4b0d-4bef-9ce6-41cecadf4a4c None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:52.662593 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5582f8b7-4b0d-4bef-9ce6-41cecadf4a4c None admin] PATH_INFO: `/v3/users/nova` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:52.677530 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5582f8b7-4b0d-4bef-9ce6-41cecadf4a4c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:52.680730 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5582f8b7-4b0d-4bef-9ce6-41cecadf4a4c None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:52.704050 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-5582f8b7-4b0d-4bef-9ce6-41cecadf4a4c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 06 19:59:52.704050 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:52.704050 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:52.704050 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:52.704050 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:52.704050 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:52.704050 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:52.704050 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:52.704050 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:52.704050 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:52.704050 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:52.704050 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:52.704050 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 19:59:52.704050 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 19:59:52.704050 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 19:59:52.704050 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 19:59:52.706309 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:52.706309 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:52.706309 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 19:59:52.706309 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 19:59:52.706309 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 19:59:52.706309 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 19:59:52.706309 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:52.706309 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:52.706309 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:52.706309 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:52.706309 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:52.706309 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:52.706309 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:52.706309 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:52.706309 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:52.706309 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:52.706309 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:52.706309 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:52.708714 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:52.708714 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:52.708714 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 19:59:52.708714 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 19:59:52.708714 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 19:59:52.708714 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 19:59:52.708714 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 19:59:52.708714 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 19:59:52.708714 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 06 19:59:52.708714 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 19:59:52.708714 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 367/709] 10.222.0.20 () {56 vars in 1093 bytes} [Tue Sep 6 19:59:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 363 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:59:52.719690 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92df328-7fc1-486f-aa6a-c539fb9421ef None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:52.758387 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5582f8b7-4b0d-4bef-9ce6-41cecadf4a4c None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:52.864576 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:52.927067 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92df328-7fc1-486f-aa6a-c539fb9421ef None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:52.938392 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5582f8b7-4b0d-4bef-9ce6-41cecadf4a4c None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:52.982740 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb958f33-5909-4e2f-ad0f-b253a4292c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:52.982740 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb958f33-5909-4e2f-ad0f-b253a4292c6b None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:52.982740 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb958f33-5909-4e2f-ad0f-b253a4292c6b None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:52.982740 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb958f33-5909-4e2f-ad0f-b253a4292c6b None admin] PATH_INFO: `/v3/users` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:52.983408 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb958f33-5909-4e2f-ad0f-b253a4292c6b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:52.984017 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb958f33-5909-4e2f-ad0f-b253a4292c6b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:52.985205 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb958f33-5909-4e2f-ad0f-b253a4292c6b None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:52.988811 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:53.018454 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 343/710] 10.222.0.20 () {56 vars in 1133 bytes} [Tue Sep 6 19:59:52 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:53.018454 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf7739c-af90-4af3-a382-1e462589063a None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:53.018454 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf7739c-af90-4af3-a382-1e462589063a None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:53.018454 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf7739c-af90-4af3-a382-1e462589063a None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:53.018454 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 344/711] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 19:59:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 19:59:53.026390 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf7739c-af90-4af3-a382-1e462589063a None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:53.034963 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a8905a-2e65-4c7f-a98f-e839f9c0fa43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:53.035656 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a8905a-2e65-4c7f-a98f-e839f9c0fa43 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:53.037110 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a8905a-2e65-4c7f-a98f-e839f9c0fa43 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:53.037406 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a8905a-2e65-4c7f-a98f-e839f9c0fa43 None admin] PATH_INFO: `/v3/services/network` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:53.038196 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a8905a-2e65-4c7f-a98f-e839f9c0fa43 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:53.039747 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a8905a-2e65-4c7f-a98f-e839f9c0fa43 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:53.055877 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-81a8905a-2e65-4c7f-a98f-e839f9c0fa43 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 06 19:59:53.055877 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:53.055877 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 19:59:53.055877 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 19:59:53.055877 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 19:59:53.055877 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 19:59:53.055877 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 19:59:53.055877 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 19:59:53.055877 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 06 19:59:53.055877 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 19:59:53.055877 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 19:59:53.055877 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 19:59:53.055877 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:53.055877 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:53.055877 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:53.055877 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:53.055877 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:53.058704 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:53.058704 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:53.058704 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:53.058704 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:53.058704 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:53.058704 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:53.058704 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 19:59:53.058704 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 19:59:53.058704 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 19:59:53.058704 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 19:59:53.058704 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:53.058704 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:53.058704 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:53.058704 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:53.058704 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:53.058704 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:53.060187 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:53.060187 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:53.060187 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:53.060187 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:53.060187 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:53.060187 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:53.060187 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:53.060187 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:53.060187 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:53.060187 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:53.060187 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 19:59:53.060187 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 19:59:53.060187 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 06 19:59:53.060187 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 19:59:53.063089 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 368/712] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 6 19:59:52 2022] GET /identity/v3/services/network => generated 88 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:59:53.064132 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cf0314-bf30-46e0-81c5-bea9840e0b03 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:53.064454 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cf0314-bf30-46e0-81c5-bea9840e0b03 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:53.064742 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cf0314-bf30-46e0-81c5-bea9840e0b03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:53.090846 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-55cf0314-bf30-46e0-81c5-bea9840e0b03 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 19:59:53.092140 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-55cf0314-bf30-46e0-81c5-bea9840e0b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 19:59:53.205748 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 369/713] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 19:59:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 19:59:53.205748 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cf0314-bf30-46e0-81c5-bea9840e0b03 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:53.241936 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf7739c-af90-4af3-a382-1e462589063a None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:53.297777 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:53.320290 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07528a14-b05e-4825-a750-ad82ebe96eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:53.321203 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07528a14-b05e-4825-a750-ad82ebe96eb9 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:53.321203 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07528a14-b05e-4825-a750-ad82ebe96eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:53.321480 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07528a14-b05e-4825-a750-ad82ebe96eb9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:53.419958 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cf0314-bf30-46e0-81c5-bea9840e0b03 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:53.474646 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:53.490482 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07528a14-b05e-4825-a750-ad82ebe96eb9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:53.492220 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07528a14-b05e-4825-a750-ad82ebe96eb9 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:53.495023 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aef302-4833-4ea8-be8f-f3291fe495ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:53.496086 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aef302-4833-4ea8-be8f-f3291fe495ba None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:53.496848 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aef302-4833-4ea8-be8f-f3291fe495ba None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:53.497525 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aef302-4833-4ea8-be8f-f3291fe495ba None admin] PATH_INFO: `/v3/services` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:53.498521 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66aef302-4833-4ea8-be8f-f3291fe495ba None admin] RBAC: Adding query filter params (name=network) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:53.499074 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66aef302-4833-4ea8-be8f-f3291fe495ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:53.501855 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66aef302-4833-4ea8-be8f-f3291fe495ba None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:53.510544 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 370/714] 10.222.0.20 () {56 vars in 1112 bytes} [Tue Sep 6 19:59:53 2022] GET /identity/v3/services?name=network => generated 124 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:53.519840 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aef302-4833-4ea8-be8f-f3291fe495ba None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:53.712667 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 345/715] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 6 19:59:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 687 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 19:59:53.715205 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a469aa-a332-47b5-92b2-a063805cf064 None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:53.715726 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a469aa-a332-47b5-92b2-a063805cf064 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:53.716298 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a469aa-a332-47b5-92b2-a063805cf064 None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:53.719612 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 346/716] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 19:59:53 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 19:59:53.721097 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a469aa-a332-47b5-92b2-a063805cf064 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:53.732168 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aef302-4833-4ea8-be8f-f3291fe495ba None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:53.770181 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:53.788634 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b15ba0-fa2c-402b-a314-2d9976daa01b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:53.789314 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b15ba0-fa2c-402b-a314-2d9976daa01b None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:53.789630 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b15ba0-fa2c-402b-a314-2d9976daa01b None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:53.789938 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b15ba0-fa2c-402b-a314-2d9976daa01b None admin] PATH_INFO: `/v3/services` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:53.790604 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b15ba0-fa2c-402b-a314-2d9976daa01b None admin] RBAC: Adding query filter params (type=network) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:53.790917 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b15ba0-fa2c-402b-a314-2d9976daa01b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:53.792337 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b15ba0-fa2c-402b-a314-2d9976daa01b None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:53.798084 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 371/717] 10.222.0.20 () {56 vars in 1112 bytes} [Tue Sep 6 19:59:53 2022] GET /identity/v3/services?type=network => generated 124 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:53.800220 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745e025f-205c-4e2c-b454-ac6aac151661 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:53.800517 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745e025f-205c-4e2c-b454-ac6aac151661 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:53.800781 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745e025f-205c-4e2c-b454-ac6aac151661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:53.821481 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-745e025f-205c-4e2c-b454-ac6aac151661 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 19:59:53.822819 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-745e025f-205c-4e2c-b454-ac6aac151661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 19:59:53.877515 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a469aa-a332-47b5-92b2-a063805cf064 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:53.917949 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 372/718] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 19:59:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 19:59:53.919059 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745e025f-205c-4e2c-b454-ac6aac151661 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:53.920745 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef2b3bc-6656-429f-8b42-88d8dbd910ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:53.921559 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef2b3bc-6656-429f-8b42-88d8dbd910ff None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:53.921909 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef2b3bc-6656-429f-8b42-88d8dbd910ff None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:53.922184 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef2b3bc-6656-429f-8b42-88d8dbd910ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:53.926021 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:53.938584 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef2b3bc-6656-429f-8b42-88d8dbd910ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:53.941628 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef2b3bc-6656-429f-8b42-88d8dbd910ff None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:53.977291 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-0ef2b3bc-6656-429f-8b42-88d8dbd910ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 19:59:53.977291 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:53.977291 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:53.977291 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:53.977291 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:53.977291 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:53.977291 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:53.977291 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:53.977291 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:53.977291 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:53.977291 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:53.977291 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:53.977291 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 19:59:53.977291 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 19:59:53.977291 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 19:59:53.977291 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 19:59:53.979241 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:53.979241 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:53.979241 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:53.979241 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:53.979241 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:53.979241 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:53.979241 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:53.979241 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:53.979241 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:53.979241 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:53.979241 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:53.979241 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:53.979241 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:53.979241 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:53.979241 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:53.979241 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:53.979241 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 19:59:53.981076 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 19:59:53.981076 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 19:59:53.981076 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 19:59:53.981076 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 347/719] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 19:59:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:59:53.993752 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef2b3bc-6656-429f-8b42-88d8dbd910ff None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:54.180006 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745e025f-205c-4e2c-b454-ac6aac151661 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:54.196266 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef2b3bc-6656-429f-8b42-88d8dbd910ff None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:54.200338 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:54.257914 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74855dac-75f7-4a30-b8c9-e310ef39d25e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:54.260902 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74855dac-75f7-4a30-b8c9-e310ef39d25e None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:54.261491 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74855dac-75f7-4a30-b8c9-e310ef39d25e None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:54.262023 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74855dac-75f7-4a30-b8c9-e310ef39d25e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:54.294699 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74855dac-75f7-4a30-b8c9-e310ef39d25e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 19:59:54.296578 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff44e6ef-50a7-4de0-bb16-f25586ac6b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:54.297667 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff44e6ef-50a7-4de0-bb16-f25586ac6b77 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:54.298190 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff44e6ef-50a7-4de0-bb16-f25586ac6b77 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:54.299369 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff44e6ef-50a7-4de0-bb16-f25586ac6b77 None admin] PATH_INFO: `/v3/domains` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:54.300587 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff44e6ef-50a7-4de0-bb16-f25586ac6b77 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:54.301239 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff44e6ef-50a7-4de0-bb16-f25586ac6b77 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:54.303366 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff44e6ef-50a7-4de0-bb16-f25586ac6b77 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:54.303839 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-74855dac-75f7-4a30-b8c9-e310ef39d25e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 19:59:54.303839 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:54.303839 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:54.303839 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:54.303839 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:54.303839 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:54.303839 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:54.303839 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:54.303839 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:54.303839 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:54.303839 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:54.303839 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:54.303839 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 19:59:54.303839 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 19:59:54.303839 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 19:59:54.303839 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 19:59:54.305340 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 19:59:54.305340 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 19:59:54.305340 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 19:59:54.305340 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 19:59:54.305340 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:54.305340 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:54.305340 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:54.305340 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:54.305340 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:54.305340 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:54.305340 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:54.305340 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:54.305340 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:54.305340 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:54.305340 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:54.305340 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:54.306508 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:54.306508 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:54.306508 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:54.306508 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:54.306508 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 19:59:54.306508 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 19:59:54.306508 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 19:59:54.306508 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 19:59:54.306508 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 19:59:54.306508 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 19:59:54.306508 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 19:59:54.306508 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 19:59:54.306508 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 373/720] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Sep 6 19:59:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 387 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:59:54.306508 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74855dac-75f7-4a30-b8c9-e310ef39d25e None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:54.327398 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 348/721] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 6 19:59:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:54.328951 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff44e6ef-50a7-4de0-bb16-f25586ac6b77 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:54.378025 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:54.441189 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74855dac-75f7-4a30-b8c9-e310ef39d25e None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:54.487345 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3419defc-d445-4676-9a85-9f0ad0d15a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:54.488104 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3419defc-d445-4676-9a85-9f0ad0d15a50 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:54.488394 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3419defc-d445-4676-9a85-9f0ad0d15a50 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:54.488670 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3419defc-d445-4676-9a85-9f0ad0d15a50 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:54.501573 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3419defc-d445-4676-9a85-9f0ad0d15a50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 19:59:54.506953 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-3419defc-d445-4676-9a85-9f0ad0d15a50 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 19:59:54.506953 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:54.506953 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:54.506953 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:54.506953 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:54.506953 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:54.506953 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:54.506953 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:54.506953 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:54.506953 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:54.506953 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:54.506953 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:54.506953 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 19:59:54.506953 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 19:59:54.506953 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 19:59:54.506953 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 19:59:54.508912 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 19:59:54.508912 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 19:59:54.508912 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 19:59:54.508912 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 19:59:54.508912 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:54.508912 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:54.508912 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:54.508912 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:54.508912 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:54.508912 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:54.508912 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:54.508912 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:54.508912 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:54.508912 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:54.508912 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:54.508912 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:54.510632 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:54.510632 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:54.510632 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:54.510632 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:54.510632 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 19:59:54.510632 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 19:59:54.510632 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 19:59:54.510632 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 19:59:54.510632 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 19:59:54.510632 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 19:59:54.510632 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 19:59:54.510632 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 19:59:54.513885 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 374/722] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Sep 6 19:59:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:59:54.514784 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3419defc-d445-4676-9a85-9f0ad0d15a50 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:54.520949 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff44e6ef-50a7-4de0-bb16-f25586ac6b77 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:54.552451 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:54.607879 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94a6206-29c4-421e-88a6-4c4b714d5f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:54.610552 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94a6206-29c4-421e-88a6-4c4b714d5f92 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:54.611809 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94a6206-29c4-421e-88a6-4c4b714d5f92 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:54.612559 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94a6206-29c4-421e-88a6-4c4b714d5f92 None admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:54.616229 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94a6206-29c4-421e-88a6-4c4b714d5f92 None admin] RBAC: Adding query filter params (name=service) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:54.617153 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94a6206-29c4-421e-88a6-4c4b714d5f92 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:54.620167 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94a6206-29c4-421e-88a6-4c4b714d5f92 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:54.650092 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 349/723] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Sep 6 19:59:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:54.652310 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94a6206-29c4-421e-88a6-4c4b714d5f92 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:54.655349 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3419defc-d445-4676-9a85-9f0ad0d15a50 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:54.702164 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc444614-5da5-458a-9709-87b28a10592d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:54.703128 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc444614-5da5-458a-9709-87b28a10592d None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:54.703291 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc444614-5da5-458a-9709-87b28a10592d None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:54.703610 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc444614-5da5-458a-9709-87b28a10592d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:54.712619 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc444614-5da5-458a-9709-87b28a10592d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:54.714174 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc444614-5da5-458a-9709-87b28a10592d None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:54.728358 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-cc444614-5da5-458a-9709-87b28a10592d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 19:59:54.728358 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:54.728358 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:54.728358 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:54.728358 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:54.728358 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:54.728358 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:54.728358 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:54.728358 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:54.728358 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:54.728358 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:54.728358 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:54.728358 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 19:59:54.728358 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 19:59:54.728358 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 19:59:54.730489 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 19:59:54.730489 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:54.730489 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:54.730489 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:54.730489 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:54.730489 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:54.730489 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:54.730489 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:54.730489 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:54.730489 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:54.730489 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:54.730489 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:54.730489 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:54.730489 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:54.730489 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:54.730489 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:54.730489 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:54.732448 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 19:59:54.732448 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 19:59:54.732448 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 19:59:54.732448 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 19:59:54.732448 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 19:59:54.732448 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 19:59:54.732448 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 19:59:54.732448 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 19:59:54.735397 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 375/724] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 6 19:59:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:59:54.736551 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc444614-5da5-458a-9709-87b28a10592d None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:54.814752 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:54.855815 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:54.881191 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc444614-5da5-458a-9709-87b28a10592d None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:54.886466 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94a6206-29c4-421e-88a6-4c4b714d5f92 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:54.942140 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73ec5de-f104-4496-aada-df1ee49be1f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:54.943005 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73ec5de-f104-4496-aada-df1ee49be1f6 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:54.943557 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73ec5de-f104-4496-aada-df1ee49be1f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:54.944312 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73ec5de-f104-4496-aada-df1ee49be1f6 None admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:54.944632 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73ec5de-f104-4496-aada-df1ee49be1f6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:54.944947 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73ec5de-f104-4496-aada-df1ee49be1f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:54.946501 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73ec5de-f104-4496-aada-df1ee49be1f6 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:54.949401 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e167abb9-4f24-493b-8f13-bf92a66238f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:54.950330 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e167abb9-4f24-493b-8f13-bf92a66238f9 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:54.950686 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e167abb9-4f24-493b-8f13-bf92a66238f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:54.951042 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e167abb9-4f24-493b-8f13-bf92a66238f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:54.961440 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 350/725] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 6 19:59:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:54.963321 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73ec5de-f104-4496-aada-df1ee49be1f6 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:54.963495 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e167abb9-4f24-493b-8f13-bf92a66238f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:54.965182 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e167abb9-4f24-493b-8f13-bf92a66238f9 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:54.982296 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-e167abb9-4f24-493b-8f13-bf92a66238f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 19:59:54.982296 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:54.982296 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:54.982296 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:54.982296 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:54.982296 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:54.982296 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:54.982296 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:54.982296 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:54.982296 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:54.982296 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:54.982296 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:54.982296 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 19:59:54.982296 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 19:59:54.982296 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 19:59:54.982296 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 19:59:54.983536 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:54.983536 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:54.983536 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:54.983536 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:54.983536 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:54.983536 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:54.983536 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:54.983536 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:54.983536 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:54.983536 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:54.983536 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:54.983536 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:54.983536 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:54.983536 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:54.983536 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:54.983536 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:54.983536 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 19:59:54.984889 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 19:59:54.984889 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 19:59:54.984889 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 19:59:54.989338 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 376/726] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 19:59:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:59:54.990916 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e89918d-7c79-4cee-90e4-0f7053c5e769 None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:54.991556 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e89918d-7c79-4cee-90e4-0f7053c5e769 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:54.991556 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e89918d-7c79-4cee-90e4-0f7053c5e769 None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:54.997766 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 377/727] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 19:59:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 19:59:54.997766 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e89918d-7c79-4cee-90e4-0f7053c5e769 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:55.114968 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73ec5de-f104-4496-aada-df1ee49be1f6 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:55.138660 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e89918d-7c79-4cee-90e4-0f7053c5e769 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:55.176671 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:55.189671 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f3e329-e33e-4f2a-8bec-652a40ce5779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:55.190350 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f3e329-e33e-4f2a-8bec-652a40ce5779 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:55.190851 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f3e329-e33e-4f2a-8bec-652a40ce5779 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:55.191275 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f3e329-e33e-4f2a-8bec-652a40ce5779 None admin] PATH_INFO: `/v3/projects` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:55.192088 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f3e329-e33e-4f2a-8bec-652a40ce5779 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:55.192403 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f3e329-e33e-4f2a-8bec-652a40ce5779 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:55.193932 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f3e329-e33e-4f2a-8bec-652a40ce5779 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:55.195299 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827c2375-c18d-4003-9bb6-2d5db2fec8e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:55.196412 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827c2375-c18d-4003-9bb6-2d5db2fec8e3 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:55.196905 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827c2375-c18d-4003-9bb6-2d5db2fec8e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:55.197391 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827c2375-c18d-4003-9bb6-2d5db2fec8e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:55.213669 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 351/728] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Sep 6 19:59:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:55.213669 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f3e329-e33e-4f2a-8bec-652a40ce5779 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:55.214360 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:55.222629 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827c2375-c18d-4003-9bb6-2d5db2fec8e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:55.225796 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827c2375-c18d-4003-9bb6-2d5db2fec8e3 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:55.258164 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-827c2375-c18d-4003-9bb6-2d5db2fec8e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 19:59:55.258164 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:55.258164 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:55.258164 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:55.258164 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:55.258164 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:55.258164 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:55.258164 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:55.258164 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:55.258164 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:55.258164 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:55.258164 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:55.258164 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 19:59:55.258164 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 19:59:55.258164 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 19:59:55.258164 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 19:59:55.260043 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:55.260043 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:55.260043 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:55.260043 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:55.260043 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:55.260043 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:55.260043 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:55.260043 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:55.260043 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:55.260043 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:55.260043 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:55.260043 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:55.260043 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:55.260043 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:55.260043 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:55.260043 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:55.260043 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 19:59:55.262665 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 19:59:55.262665 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 19:59:55.262665 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 19:59:55.262665 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 378/729] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 19:59:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:59:55.263130 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518ff50d-7c47-4fb4-88ba-b1f849303643 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:55.263701 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518ff50d-7c47-4fb4-88ba-b1f849303643 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:55.264021 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518ff50d-7c47-4fb4-88ba-b1f849303643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:55.296488 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-518ff50d-7c47-4fb4-88ba-b1f849303643 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 19:59:55.299137 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-518ff50d-7c47-4fb4-88ba-b1f849303643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 19:59:55.410875 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f3e329-e33e-4f2a-8bec-652a40ce5779 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:55.441660 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 379/730] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 19:59:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 19:59:55.441660 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518ff50d-7c47-4fb4-88ba-b1f849303643 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:55.477316 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b076eb8-28d9-4c48-b86d-da879203fcd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:55.478566 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b076eb8-28d9-4c48-b86d-da879203fcd4 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:55.479152 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b076eb8-28d9-4c48-b86d-da879203fcd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:55.479717 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b076eb8-28d9-4c48-b86d-da879203fcd4 None admin] PATH_INFO: `/v3/domains` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:55.480752 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b076eb8-28d9-4c48-b86d-da879203fcd4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:55.481402 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b076eb8-28d9-4c48-b86d-da879203fcd4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:55.484051 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b076eb8-28d9-4c48-b86d-da879203fcd4 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:55.515190 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 352/731] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 6 19:59:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:55.517243 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b076eb8-28d9-4c48-b86d-da879203fcd4 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:55.633825 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:55.670123 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518ff50d-7c47-4fb4-88ba-b1f849303643 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:55.719051 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:55.742516 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6e3142-87d6-4ee9-b768-08586485b6a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:55.743991 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6e3142-87d6-4ee9-b768-08586485b6a0 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:55.744766 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b076eb8-28d9-4c48-b86d-da879203fcd4 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:55.745048 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6e3142-87d6-4ee9-b768-08586485b6a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:55.745825 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6e3142-87d6-4ee9-b768-08586485b6a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:55.747860 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6e3142-87d6-4ee9-b768-08586485b6a0 None admin] RBAC: Adding query filter params (role.id=e0ab41d8ae22474688b5361bcc75e0f0, scope.project.id=6aef43493bb1420a9e122313d8db7d5a, user.id=5776cbf4f6714b2083c3ebdd6d98cc87) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:55.748763 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6e3142-87d6-4ee9-b768-08586485b6a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:55.752191 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6e3142-87d6-4ee9-b768-08586485b6a0 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:55.764480 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 380/732] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Sep 6 19:59:55 2022] GET /identity/v3/role_assignments?user.id=5776cbf4f6714b2083c3ebdd6d98cc87&scope.project.id=6aef43493bb1420a9e122313d8db7d5a&role.id=e0ab41d8ae22474688b5361bcc75e0f0 => generated 259 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:55.767506 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6e3142-87d6-4ee9-b768-08586485b6a0 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:55.872096 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c7bcf9-33c6-4e6c-ab60-fd3eca4404bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:55.874367 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c7bcf9-33c6-4e6c-ab60-fd3eca4404bf None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:55.875369 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c7bcf9-33c6-4e6c-ab60-fd3eca4404bf None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:55.876845 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c7bcf9-33c6-4e6c-ab60-fd3eca4404bf None admin] PATH_INFO: `/v3/domains` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:55.880885 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c7bcf9-33c6-4e6c-ab60-fd3eca4404bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:55.881826 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c7bcf9-33c6-4e6c-ab60-fd3eca4404bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:55.887146 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c7bcf9-33c6-4e6c-ab60-fd3eca4404bf None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:55.939015 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 353/733] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 6 19:59:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 423 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:55.949193 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c7bcf9-33c6-4e6c-ab60-fd3eca4404bf None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:56.057851 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6e3142-87d6-4ee9-b768-08586485b6a0 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:56.113214 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7cf23e-d2b3-4fdc-8ef6-abd5226bdb31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:56.114166 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7cf23e-d2b3-4fdc-8ef6-abd5226bdb31 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:56.114455 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7cf23e-d2b3-4fdc-8ef6-abd5226bdb31 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:56.114810 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7cf23e-d2b3-4fdc-8ef6-abd5226bdb31 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:56.124162 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7cf23e-d2b3-4fdc-8ef6-abd5226bdb31 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:56.125950 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7cf23e-d2b3-4fdc-8ef6-abd5226bdb31 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:56.140587 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-6e7cf23e-d2b3-4fdc-8ef6-abd5226bdb31 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 06 19:59:56.140587 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:56.140587 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:56.140587 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:56.140587 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:56.140587 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:56.140587 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:56.140587 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:56.140587 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:56.140587 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:56.140587 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:56.140587 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:56.140587 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 19:59:56.140587 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 19:59:56.140587 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 19:59:56.140587 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 19:59:56.141757 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:56.141757 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:56.141757 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 19:59:56.141757 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 19:59:56.141757 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 19:59:56.141757 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 19:59:56.141757 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:56.141757 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:56.141757 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:56.141757 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:56.141757 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:56.141757 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:56.141757 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:56.141757 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:56.141757 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:56.141757 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:56.141757 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:56.141757 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:56.142828 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:56.142828 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:56.142828 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 19:59:56.142828 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 19:59:56.142828 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 19:59:56.142828 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 19:59:56.142828 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 19:59:56.142828 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 19:59:56.142828 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 06 19:59:56.142828 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 19:59:56.142828 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 381/734] 10.222.0.20 () {56 vars in 1117 bytes} [Tue Sep 6 19:59:55 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 377 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:59:56.143537 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7cf23e-d2b3-4fdc-8ef6-abd5226bdb31 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:56.171007 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c7bcf9-33c6-4e6c-ab60-fd3eca4404bf None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:56.191010 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:56.200766 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:56.241894 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b254106-003d-43e5-9131-07d2d6325e3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:56.242980 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b254106-003d-43e5-9131-07d2d6325e3b None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:56.243454 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b254106-003d-43e5-9131-07d2d6325e3b None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:56.243922 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b254106-003d-43e5-9131-07d2d6325e3b None admin] PATH_INFO: `/v3/users/swift` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:56.263000 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b254106-003d-43e5-9131-07d2d6325e3b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:56.267402 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b254106-003d-43e5-9131-07d2d6325e3b None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:56.290049 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7cf23e-d2b3-4fdc-8ef6-abd5226bdb31 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:56.296647 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-3b254106-003d-43e5-9131-07d2d6325e3b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 06 19:59:56.296647 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:56.296647 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:56.296647 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:56.296647 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:56.296647 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:56.296647 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:56.296647 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:56.296647 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:56.296647 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:56.296647 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:56.296647 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:56.296647 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 19:59:56.296647 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 19:59:56.296647 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 19:59:56.296647 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 19:59:56.297783 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:56.297783 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:56.297783 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 19:59:56.297783 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 19:59:56.297783 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 19:59:56.297783 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 19:59:56.297783 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:56.297783 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:56.297783 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:56.297783 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:56.297783 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:56.297783 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:56.297783 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:56.297783 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:56.297783 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:56.297783 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:56.297783 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:56.297783 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:56.298808 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:56.298808 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:56.298808 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 19:59:56.298808 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 19:59:56.298808 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 19:59:56.298808 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 19:59:56.298808 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 19:59:56.298808 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 19:59:56.298808 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 06 19:59:56.298808 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 19:59:56.298808 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 354/735] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Sep 6 19:59:55 2022] GET /identity/v3/users/swift => generated 83 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:59:56.299570 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b254106-003d-43e5-9131-07d2d6325e3b None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:56.350157 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3437c18-af5c-4f6e-b879-180912fea9b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:56.351164 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3437c18-af5c-4f6e-b879-180912fea9b4 None admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:56.351667 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3437c18-af5c-4f6e-b879-180912fea9b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:56.352143 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3437c18-af5c-4f6e-b879-180912fea9b4 None admin] PATH_INFO: `/v3/services` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:56.353055 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3437c18-af5c-4f6e-b879-180912fea9b4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:56.354745 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3437c18-af5c-4f6e-b879-180912fea9b4 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:56.386182 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 382/736] 10.222.0.20 () {60 vars in 1141 bytes} [Tue Sep 6 19:59:56 2022] POST /identity/v3/services => generated 251 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 19:59:56.387931 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3437c18-af5c-4f6e-b879-180912fea9b4 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:56.445831 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b254106-003d-43e5-9131-07d2d6325e3b None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:56.507009 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac1248e-aa8b-4850-b4ef-59fab6dd4742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:56.507009 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac1248e-aa8b-4850-b4ef-59fab6dd4742 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:56.507348 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac1248e-aa8b-4850-b4ef-59fab6dd4742 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:56.507609 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac1248e-aa8b-4850-b4ef-59fab6dd4742 None admin] PATH_INFO: `/v3/users` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:56.508326 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac1248e-aa8b-4850-b4ef-59fab6dd4742 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:56.508643 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac1248e-aa8b-4850-b4ef-59fab6dd4742 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:56.510284 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac1248e-aa8b-4850-b4ef-59fab6dd4742 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:56.532858 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 355/737] 10.222.0.20 () {56 vars in 1149 bytes} [Tue Sep 6 19:59:56 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:56.533975 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e52362-1e74-4735-93b0-dadebd302052 None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:56.534275 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e52362-1e74-4735-93b0-dadebd302052 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:56.534603 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e52362-1e74-4735-93b0-dadebd302052 None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:56.536476 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 356/738] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 19:59:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 19:59:56.542982 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e52362-1e74-4735-93b0-dadebd302052 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:56.552108 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3437c18-af5c-4f6e-b879-180912fea9b4 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:56.557226 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:56.596469 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:56.655254 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c172b1ef-2c46-46e9-bf82-5ac5ad34e5fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:56.655254 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c172b1ef-2c46-46e9-bf82-5ac5ad34e5fd None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:56.655254 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c172b1ef-2c46-46e9-bf82-5ac5ad34e5fd None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:56.655254 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c172b1ef-2c46-46e9-bf82-5ac5ad34e5fd None admin] PATH_INFO: `/v3/users` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:56.655913 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c172b1ef-2c46-46e9-bf82-5ac5ad34e5fd None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:56.655913 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c172b1ef-2c46-46e9-bf82-5ac5ad34e5fd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:56.656284 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c172b1ef-2c46-46e9-bf82-5ac5ad34e5fd None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:56.692412 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 383/739] 10.222.0.20 () {56 vars in 1135 bytes} [Tue Sep 6 19:59:56 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:56.694867 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5890e8f4-c886-4ca2-b450-362bc86892f1 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:56.694867 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5890e8f4-c886-4ca2-b450-362bc86892f1 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:56.695338 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5890e8f4-c886-4ca2-b450-362bc86892f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:56.728487 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-5890e8f4-c886-4ca2-b450-362bc86892f1 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 19:59:56.730324 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-5890e8f4-c886-4ca2-b450-362bc86892f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 19:59:56.759079 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e52362-1e74-4735-93b0-dadebd302052 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:56.833603 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def0bf76-0d0d-4a56-b919-e916a980254f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:56.835503 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def0bf76-0d0d-4a56-b919-e916a980254f None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:56.836277 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def0bf76-0d0d-4a56-b919-e916a980254f None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:56.837005 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def0bf76-0d0d-4a56-b919-e916a980254f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:56.873680 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 384/740] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 19:59:56 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 19:59:56.873680 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5890e8f4-c886-4ca2-b450-362bc86892f1 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:57.067739 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def0bf76-0d0d-4a56-b919-e916a980254f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:57.070566 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def0bf76-0d0d-4a56-b919-e916a980254f None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:57.144890 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:57.163777 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:57.194559 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5890e8f4-c886-4ca2-b450-362bc86892f1 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:57.246747 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 357/741] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 6 19:59:56 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 704 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 19:59:57.255245 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def0bf76-0d0d-4a56-b919-e916a980254f None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:57.285562 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fad0f9f-fc5c-41da-b729-628c141a8851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:57.286248 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fad0f9f-fc5c-41da-b729-628c141a8851 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:57.286530 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fad0f9f-fc5c-41da-b729-628c141a8851 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:57.286815 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fad0f9f-fc5c-41da-b729-628c141a8851 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:57.381665 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fad0f9f-fc5c-41da-b729-628c141a8851 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:57.381665 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fad0f9f-fc5c-41da-b729-628c141a8851 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:57.388348 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def0bf76-0d0d-4a56-b919-e916a980254f None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:57.430792 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:57.436500 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72be9bc-5087-4669-b520-9675f194932b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:57.437651 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72be9bc-5087-4669-b520-9675f194932b None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:57.438228 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72be9bc-5087-4669-b520-9675f194932b None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:57.438857 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72be9bc-5087-4669-b520-9675f194932b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:57.460925 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72be9bc-5087-4669-b520-9675f194932b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:57.463885 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72be9bc-5087-4669-b520-9675f194932b None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:57.495730 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-e72be9bc-5087-4669-b520-9675f194932b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 19:59:57.495730 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:57.495730 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:57.495730 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:57.495730 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:57.495730 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:57.495730 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:57.495730 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:57.495730 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:57.495730 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:57.495730 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:57.495730 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:57.495730 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 19:59:57.495730 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 19:59:57.495730 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 19:59:57.495730 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 19:59:57.496748 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:57.496748 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:57.496748 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:57.496748 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:57.496748 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:57.496748 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:57.496748 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:57.496748 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:57.496748 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:57.496748 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:57.496748 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:57.496748 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:57.496748 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:57.496748 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:57.496748 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:57.496748 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:57.496748 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 19:59:57.497765 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 19:59:57.497765 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 19:59:57.497765 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 19:59:57.502419 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:57.503407 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 358/742] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 19:59:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:59:57.505078 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72be9bc-5087-4669-b520-9675f194932b None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:57.518241 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 385/743] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 6 19:59:56 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 660 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 19:59:57.519386 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fad0f9f-fc5c-41da-b729-628c141a8851 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:57.653205 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fad0f9f-fc5c-41da-b729-628c141a8851 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:57.698768 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0408ea-69e2-49e5-958a-ecc8d0b14bd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:57.699694 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0408ea-69e2-49e5-958a-ecc8d0b14bd7 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:57.700182 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0408ea-69e2-49e5-958a-ecc8d0b14bd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:57.700648 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0408ea-69e2-49e5-958a-ecc8d0b14bd7 None admin] PATH_INFO: `/v3/domains` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:57.701509 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0408ea-69e2-49e5-958a-ecc8d0b14bd7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:57.702040 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0408ea-69e2-49e5-958a-ecc8d0b14bd7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:57.703691 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0408ea-69e2-49e5-958a-ecc8d0b14bd7 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:57.718686 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 386/744] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 6 19:59:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:57.720477 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0408ea-69e2-49e5-958a-ecc8d0b14bd7 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:57.743205 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72be9bc-5087-4669-b520-9675f194932b None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:57.790916 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877fda24-ea42-4fe9-8f8f-02a289a82615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:57.791829 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877fda24-ea42-4fe9-8f8f-02a289a82615 None admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:57.792370 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877fda24-ea42-4fe9-8f8f-02a289a82615 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:57.792807 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877fda24-ea42-4fe9-8f8f-02a289a82615 None admin] PATH_INFO: `/v3/services` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:57.793858 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877fda24-ea42-4fe9-8f8f-02a289a82615 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:57.795652 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877fda24-ea42-4fe9-8f8f-02a289a82615 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:57.824265 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 359/745] 10.222.0.20 () {60 vars in 1141 bytes} [Tue Sep 6 19:59:57 2022] POST /identity/v3/services => generated 240 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 19:59:57.826717 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877fda24-ea42-4fe9-8f8f-02a289a82615 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:57.843558 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:57.913756 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:57.974482 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0408ea-69e2-49e5-958a-ecc8d0b14bd7 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:58.044068 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a284d2-f93e-4cfa-9c72-98aae98bbb27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:58.045943 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a284d2-f93e-4cfa-9c72-98aae98bbb27 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:58.046806 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a284d2-f93e-4cfa-9c72-98aae98bbb27 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:58.048165 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a284d2-f93e-4cfa-9c72-98aae98bbb27 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:58.071327 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a284d2-f93e-4cfa-9c72-98aae98bbb27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:58.075161 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a284d2-f93e-4cfa-9c72-98aae98bbb27 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:58.107003 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877fda24-ea42-4fe9-8f8f-02a289a82615 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:58.108852 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-f8a284d2-f93e-4cfa-9c72-98aae98bbb27 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 19:59:58.108852 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:58.108852 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:58.108852 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:58.108852 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:58.108852 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:58.108852 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:58.108852 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:58.108852 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:58.108852 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:58.108852 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:58.108852 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:58.108852 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 19:59:58.108852 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 19:59:58.108852 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 19:59:58.108852 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 19:59:58.111266 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:58.111266 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:58.111266 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:58.111266 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:58.111266 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:58.111266 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:58.111266 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:58.111266 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:58.111266 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:58.111266 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:58.111266 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:58.111266 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:58.111266 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:58.111266 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:58.111266 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:58.111266 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:58.111266 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 19:59:58.113605 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 19:59:58.113605 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 19:59:58.113605 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 19:59:58.116965 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 387/746] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 19:59:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 397 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:59:58.127168 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a284d2-f93e-4cfa-9c72-98aae98bbb27 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:58.157263 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d07ad0-53e4-48c0-bd80-7d490d2d0397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:58.158034 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d07ad0-53e4-48c0-bd80-7d490d2d0397 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:58.158362 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d07ad0-53e4-48c0-bd80-7d490d2d0397 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:58.158677 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d07ad0-53e4-48c0-bd80-7d490d2d0397 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:58.167961 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d07ad0-53e4-48c0-bd80-7d490d2d0397 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:58.170485 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d07ad0-53e4-48c0-bd80-7d490d2d0397 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:58.185456 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-95d07ad0-53e4-48c0-bd80-7d490d2d0397 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 19:59:58.185456 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:58.185456 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:58.185456 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:58.185456 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:58.185456 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:58.185456 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:58.185456 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:58.185456 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:58.185456 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:58.185456 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:58.185456 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:58.185456 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 19:59:58.185456 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 19:59:58.185456 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 19:59:58.187242 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 19:59:58.187242 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:58.187242 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:58.187242 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:58.187242 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:58.187242 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:58.187242 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:58.187242 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:58.187242 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:58.187242 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:58.187242 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:58.187242 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:58.187242 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:58.187242 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:58.187242 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:58.187242 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:58.187242 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:58.188978 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 19:59:58.188978 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 19:59:58.188978 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 19:59:58.188978 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 19:59:58.188978 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 19:59:58.188978 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 19:59:58.188978 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 19:59:58.188978 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 19:59:58.188978 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 360/747] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 6 19:59:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 362 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:59:58.194043 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d07ad0-53e4-48c0-bd80-7d490d2d0397 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:58.319421 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:58.328450 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d07ad0-53e4-48c0-bd80-7d490d2d0397 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:58.345658 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:58.351983 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a284d2-f93e-4cfa-9c72-98aae98bbb27 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:58.379748 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c94eee8-f6ec-4ac6-800b-e5b290147060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:58.380850 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c94eee8-f6ec-4ac6-800b-e5b290147060 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:58.381455 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c94eee8-f6ec-4ac6-800b-e5b290147060 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:58.382017 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c94eee8-f6ec-4ac6-800b-e5b290147060 None admin] PATH_INFO: `/v3/projects` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:58.383298 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c94eee8-f6ec-4ac6-800b-e5b290147060 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:58.383831 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c94eee8-f6ec-4ac6-800b-e5b290147060 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:58.385700 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c94eee8-f6ec-4ac6-800b-e5b290147060 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:58.403772 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 361/748] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Sep 6 19:59:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 19:59:58.411200 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c94eee8-f6ec-4ac6-800b-e5b290147060 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:58.427895 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985c1832-0134-48cc-9fb6-65b4db7fe9a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:58.429041 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985c1832-0134-48cc-9fb6-65b4db7fe9a1 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:58.429741 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985c1832-0134-48cc-9fb6-65b4db7fe9a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:58.430363 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985c1832-0134-48cc-9fb6-65b4db7fe9a1 None admin] PATH_INFO: `/v3/domains` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:58.431910 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985c1832-0134-48cc-9fb6-65b4db7fe9a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:58.432610 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985c1832-0134-48cc-9fb6-65b4db7fe9a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:58.435719 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985c1832-0134-48cc-9fb6-65b4db7fe9a1 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:58.462380 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 388/749] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 6 19:59:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:58.472833 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985c1832-0134-48cc-9fb6-65b4db7fe9a1 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:58.546354 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c94eee8-f6ec-4ac6-800b-e5b290147060 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:58.593323 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd46bfc6-4063-4898-8089-aa0a949e052e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:58.595235 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd46bfc6-4063-4898-8089-aa0a949e052e None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:58.595960 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd46bfc6-4063-4898-8089-aa0a949e052e None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:58.596758 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd46bfc6-4063-4898-8089-aa0a949e052e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:58.598552 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd46bfc6-4063-4898-8089-aa0a949e052e None admin] RBAC: Adding query filter params (role.id=b950bf73c93e4f7ebed923ff14e39faa, scope.project.id=6aef43493bb1420a9e122313d8db7d5a, user.id=1a52117df70447dba9bbee35d083e44f) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:58.599291 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd46bfc6-4063-4898-8089-aa0a949e052e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:58.601353 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd46bfc6-4063-4898-8089-aa0a949e052e None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:58.608761 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 362/750] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Sep 6 19:59:58 2022] GET /identity/v3/role_assignments?user.id=1a52117df70447dba9bbee35d083e44f&scope.project.id=6aef43493bb1420a9e122313d8db7d5a&role.id=b950bf73c93e4f7ebed923ff14e39faa => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 19:59:58.715361 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985c1832-0134-48cc-9fb6-65b4db7fe9a1 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:58.786015 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0dbf89-64ba-4e20-b76c-08f17fae53a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:58.787129 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0dbf89-64ba-4e20-b76c-08f17fae53a7 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:58.787570 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0dbf89-64ba-4e20-b76c-08f17fae53a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:58.787990 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0dbf89-64ba-4e20-b76c-08f17fae53a7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:58.802517 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0dbf89-64ba-4e20-b76c-08f17fae53a7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:58.805336 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0dbf89-64ba-4e20-b76c-08f17fae53a7 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:58.829952 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-4e0dbf89-64ba-4e20-b76c-08f17fae53a7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 19:59:58.829952 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 19:59:58.829952 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 19:59:58.829952 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 19:59:58.829952 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 19:59:58.829952 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 19:59:58.829952 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 19:59:58.829952 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 19:59:58.829952 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 19:59:58.829952 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 19:59:58.829952 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 19:59:58.829952 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 19:59:58.829952 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 19:59:58.829952 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 19:59:58.829952 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 19:59:58.831203 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 19:59:58.831203 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 19:59:58.831203 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 19:59:58.831203 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 19:59:58.831203 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 19:59:58.831203 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 19:59:58.831203 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 19:59:58.831203 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 19:59:58.831203 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 19:59:58.831203 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 19:59:58.831203 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 19:59:58.831203 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 19:59:58.831203 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 19:59:58.831203 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 19:59:58.831203 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 19:59:58.831203 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 19:59:58.831203 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 19:59:58.832191 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 19:59:58.832191 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 19:59:58.832191 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 19:59:58.832191 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 19:59:58.832191 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 19:59:58.832191 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 19:59:58.832191 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 19:59:58.832191 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 19:59:58.832837 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 389/751] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 6 19:59:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 361 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 19:59:58.842965 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd46bfc6-4063-4898-8089-aa0a949e052e None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:58.902745 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:59.036852 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd46bfc6-4063-4898-8089-aa0a949e052e None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:59.082522 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b086fd-bab7-4be3-a5f0-e9723f2edd44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:59.083159 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b086fd-bab7-4be3-a5f0-e9723f2edd44 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:59.083440 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b086fd-bab7-4be3-a5f0-e9723f2edd44 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:59.083713 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b086fd-bab7-4be3-a5f0-e9723f2edd44 None admin] PATH_INFO: `/v3/projects` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:59.084446 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b086fd-bab7-4be3-a5f0-e9723f2edd44 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:59.084752 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b086fd-bab7-4be3-a5f0-e9723f2edd44 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:59.086365 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b086fd-bab7-4be3-a5f0-e9723f2edd44 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:59.101042 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 363/752] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Sep 6 19:59:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:59.111027 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0dbf89-64ba-4e20-b76c-08f17fae53a7 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:59.118886 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdbd9a5-3c3a-4e16-93c3-6b264c153b37 None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:59.119442 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdbd9a5-3c3a-4e16-93c3-6b264c153b37 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:59.119442 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdbd9a5-3c3a-4e16-93c3-6b264c153b37 None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:59.120862 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 364/753] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 19:59:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 19:59:59.147395 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3354bdc9-c4ad-4b1d-8336-af0b33429d59 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:59.147712 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3354bdc9-c4ad-4b1d-8336-af0b33429d59 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:59.147957 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3354bdc9-c4ad-4b1d-8336-af0b33429d59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:59.150224 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:59.182460 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-3354bdc9-c4ad-4b1d-8336-af0b33429d59 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 19:59:59.185241 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-3354bdc9-c4ad-4b1d-8336-af0b33429d59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 19:59:59.337029 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0dbf89-64ba-4e20-b76c-08f17fae53a7 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 19:59:59.341095 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 365/754] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 19:59:59 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 19:59:59.401375 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0746eefd-c191-423b-ad31-96b2aedcd7ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 19:59:59.402966 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0746eefd-c191-423b-ad31-96b2aedcd7ba None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:59.403700 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0746eefd-c191-423b-ad31-96b2aedcd7ba None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:59.404247 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0746eefd-c191-423b-ad31-96b2aedcd7ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:59.405286 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0746eefd-c191-423b-ad31-96b2aedcd7ba None admin] RBAC: Adding query filter params (role.id=e0ab41d8ae22474688b5361bcc75e0f0, scope.project.id=6aef43493bb1420a9e122313d8db7d5a, user.id=2b72516236da47e292d78de201b0552b) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 19:59:59.405846 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0746eefd-c191-423b-ad31-96b2aedcd7ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 19:59:59.408191 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0746eefd-c191-423b-ad31-96b2aedcd7ba None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 19:59:59.418639 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 390/755] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Sep 6 19:59:59 2022] GET /identity/v3/role_assignments?user.id=2b72516236da47e292d78de201b0552b&scope.project.id=6aef43493bb1420a9e122313d8db7d5a&role.id=e0ab41d8ae22474688b5361bcc75e0f0 => generated 259 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 19:59:59.721497 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3354bdc9-c4ad-4b1d-8336-af0b33429d59 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 19:59:59.749455 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 19:59:59.975241 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f69137-4f47-4ffd-8a04-746b46c13eec None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 19:59:59.976051 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f69137-4f47-4ffd-8a04-746b46c13eec None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 19:59:59.976200 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f69137-4f47-4ffd-8a04-746b46c13eec None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 19:59:59.979115 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 391/756] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 19:59:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 19:59:59.981584 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3354bdc9-c4ad-4b1d-8336-af0b33429d59 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:00.009064 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b76013-5784-4d9e-bd3e-546ede3c2c8d None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:00.009775 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b76013-5784-4d9e-bd3e-546ede3c2c8d None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:00.010408 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b76013-5784-4d9e-bd3e-546ede3c2c8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:00.076741 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-f1b76013-5784-4d9e-bd3e-546ede3c2c8d None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:00:00.080867 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-f1b76013-5784-4d9e-bd3e-546ede3c2c8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:00:00.090962 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f28c15-0755-435d-a036-af3e1907a07f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:00.092053 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f28c15-0755-435d-a036-af3e1907a07f None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:00.092547 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f28c15-0755-435d-a036-af3e1907a07f None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:00.093118 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f28c15-0755-435d-a036-af3e1907a07f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:00.136043 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f28c15-0755-435d-a036-af3e1907a07f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 20:00:00.149361 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-05f28c15-0755-435d-a036-af3e1907a07f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 20:00:00.149361 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:00.149361 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:00:00.149361 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:00:00.149361 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:00:00.149361 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:00:00.149361 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:00:00.149361 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:00:00.149361 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:00:00.149361 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:00:00.149361 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:00:00.149361 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:00:00.149361 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 20:00:00.149361 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 20:00:00.149361 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 20:00:00.149361 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 20:00:00.151223 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 20:00:00.151223 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 20:00:00.151223 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 20:00:00.151223 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 20:00:00.151223 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:00:00.151223 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:00:00.151223 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:00:00.151223 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:00:00.151223 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:00:00.151223 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:00:00.151223 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:00:00.151223 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 20:00:00.151223 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:00:00.151223 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:00:00.151223 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:00:00.151223 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:00:00.153025 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:00:00.153025 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:00:00.153025 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:00:00.153025 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:00:00.153025 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 20:00:00.153025 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 20:00:00.153025 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 20:00:00.153025 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 20:00:00.153025 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 20:00:00.153025 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 20:00:00.153025 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 20:00:00.153025 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:00:00.158963 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 366/757] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Sep 6 19:59:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 439 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:00:00.167557 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f28c15-0755-435d-a036-af3e1907a07f None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:00.282380 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:00.303899 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 392/758] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:00:00 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 20:00:00.353966 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f28c15-0755-435d-a036-af3e1907a07f None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:00.426258 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884bb7cf-303a-4436-afe4-fb7ed713ad30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:00.427406 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884bb7cf-303a-4436-afe4-fb7ed713ad30 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:00.427908 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884bb7cf-303a-4436-afe4-fb7ed713ad30 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:00.428407 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884bb7cf-303a-4436-afe4-fb7ed713ad30 None admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:00.429886 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884bb7cf-303a-4436-afe4-fb7ed713ad30 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:00.430468 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884bb7cf-303a-4436-afe4-fb7ed713ad30 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:00.433884 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884bb7cf-303a-4436-afe4-fb7ed713ad30 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:00.449353 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 367/759] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 6 20:00:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 20:00:00.459643 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b76013-5784-4d9e-bd3e-546ede3c2c8d None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:00.504159 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:00.707078 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b76013-5784-4d9e-bd3e-546ede3c2c8d None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:00.781920 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e54c49b-b90d-41e5-8a9b-f500010f3a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:00.782220 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e54c49b-b90d-41e5-8a9b-f500010f3a4e None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:00.782356 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e54c49b-b90d-41e5-8a9b-f500010f3a4e None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:00.782703 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e54c49b-b90d-41e5-8a9b-f500010f3a4e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:00.797646 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e54c49b-b90d-41e5-8a9b-f500010f3a4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:00.797646 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e54c49b-b90d-41e5-8a9b-f500010f3a4e None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:00.815064 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884bb7cf-303a-4436-afe4-fb7ed713ad30 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:00.815516 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-4e54c49b-b90d-41e5-8a9b-f500010f3a4e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 20:00:00.815516 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:00.815516 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:00:00.815516 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:00:00.815516 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:00:00.815516 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:00:00.815516 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:00:00.815516 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:00:00.815516 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:00:00.815516 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:00:00.815516 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:00:00.815516 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:00:00.815516 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 20:00:00.815516 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 20:00:00.815516 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 20:00:00.815516 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 20:00:00.817173 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:00:00.817173 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:00:00.817173 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:00:00.817173 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:00:00.817173 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:00:00.817173 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:00:00.817173 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:00:00.817173 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 20:00:00.817173 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:00:00.817173 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:00:00.817173 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:00:00.817173 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:00:00.817173 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:00:00.817173 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:00:00.817173 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:00:00.817173 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:00:00.817173 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 20:00:00.818890 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 20:00:00.818890 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 20:00:00.818890 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 20:00:00.818890 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 393/760] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 20:00:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 359 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:00:00.830952 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e54c49b-b90d-41e5-8a9b-f500010f3a4e None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:01.048595 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884bb7cf-303a-4436-afe4-fb7ed713ad30 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:01.069983 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:01.158691 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:01.161493 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedc9e36-304b-4bf1-8a0a-b8e5316d342f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:01.163530 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedc9e36-304b-4bf1-8a0a-b8e5316d342f None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:01.165210 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedc9e36-304b-4bf1-8a0a-b8e5316d342f None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:01.166101 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedc9e36-304b-4bf1-8a0a-b8e5316d342f None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:01.167808 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedc9e36-304b-4bf1-8a0a-b8e5316d342f None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:01.173591 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedc9e36-304b-4bf1-8a0a-b8e5316d342f None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:01.204302 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-bedc9e36-304b-4bf1-8a0a-b8e5316d342f None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 06 20:00:01.204302 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:01.204302 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 20:00:01.204302 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 20:00:01.204302 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 20:00:01.204302 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 20:00:01.204302 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 20:00:01.204302 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 20:00:01.204302 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 06 20:00:01.204302 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:00:01.204302 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 20:00:01.204302 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:00:01.204302 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:01.204302 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:00:01.204302 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:00:01.204302 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:00:01.204302 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:00:01.206326 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:00:01.206326 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:00:01.206326 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:00:01.206326 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:00:01.206326 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:00:01.206326 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:00:01.206326 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 20:00:01.206326 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 20:00:01.206326 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 20:00:01.206326 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 20:00:01.206326 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:00:01.206326 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:00:01.206326 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:00:01.206326 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:00:01.206326 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:00:01.206326 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:00:01.208048 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:00:01.208048 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 20:00:01.208048 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:00:01.208048 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:00:01.208048 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:00:01.208048 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:00:01.208048 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:00:01.208048 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:00:01.208048 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:00:01.208048 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:00:01.208048 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 20:00:01.208048 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 20:00:01.208048 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 06 20:00:01.208048 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:00:01.221242 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 368/761] 10.222.0.20 () {56 vars in 1129 bytes} [Tue Sep 6 20:00:00 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 408 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:00:01.228929 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedc9e36-304b-4bf1-8a0a-b8e5316d342f None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:01.233173 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e54c49b-b90d-41e5-8a9b-f500010f3a4e None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:01.426759 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33fbe09-4521-4d56-9561-81e4722d2a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:01.433265 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33fbe09-4521-4d56-9561-81e4722d2a4e None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:01.436362 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33fbe09-4521-4d56-9561-81e4722d2a4e None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:01.439297 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33fbe09-4521-4d56-9561-81e4722d2a4e None admin] PATH_INFO: `/v3/domains` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:01.442008 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33fbe09-4521-4d56-9561-81e4722d2a4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:01.446805 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33fbe09-4521-4d56-9561-81e4722d2a4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:01.456880 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33fbe09-4521-4d56-9561-81e4722d2a4e None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:01.548761 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 394/762] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 6 20:00:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 718 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:00:01.563181 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33fbe09-4521-4d56-9561-81e4722d2a4e None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:01.981083 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33fbe09-4521-4d56-9561-81e4722d2a4e None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:02.018268 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedc9e36-304b-4bf1-8a0a-b8e5316d342f None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:02.048181 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:02.071299 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c20c23d-e74f-44f1-bc7f-ae44321654bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:02.073873 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c20c23d-e74f-44f1-bc7f-ae44321654bb None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:02.075002 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c20c23d-e74f-44f1-bc7f-ae44321654bb None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:02.075847 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c20c23d-e74f-44f1-bc7f-ae44321654bb None admin] PATH_INFO: `/v3/users/nova` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:02.092721 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c20c23d-e74f-44f1-bc7f-ae44321654bb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:02.097320 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c20c23d-e74f-44f1-bc7f-ae44321654bb None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:02.126837 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-7c20c23d-e74f-44f1-bc7f-ae44321654bb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 06 20:00:02.126837 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:02.126837 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:00:02.126837 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:00:02.126837 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:00:02.126837 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:00:02.126837 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:00:02.126837 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:00:02.126837 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:00:02.126837 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:00:02.126837 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:00:02.126837 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:00:02.126837 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 20:00:02.126837 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 20:00:02.126837 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 20:00:02.126837 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 20:00:02.128747 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:00:02.128747 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:00:02.128747 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 20:00:02.128747 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 20:00:02.128747 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 20:00:02.128747 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 20:00:02.128747 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:00:02.128747 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:00:02.128747 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:00:02.128747 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:00:02.128747 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:00:02.128747 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 20:00:02.128747 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:00:02.128747 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:00:02.128747 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:00:02.128747 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:00:02.128747 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:00:02.128747 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:00:02.130725 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:00:02.130725 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:00:02.130725 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 20:00:02.130725 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 20:00:02.130725 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 20:00:02.130725 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 20:00:02.130725 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 20:00:02.130725 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 20:00:02.130725 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 06 20:00:02.130725 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 20:00:02.137062 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 395/763] 10.222.0.20 () {56 vars in 1093 bytes} [Tue Sep 6 20:00:01 2022] GET /identity/v3/users/nova => generated 82 bytes in 576 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:00:02.148428 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c20c23d-e74f-44f1-bc7f-ae44321654bb None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:02.150245 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf9d11d-573b-4cca-9e5d-26e244088816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:02.150890 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf9d11d-573b-4cca-9e5d-26e244088816 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:02.151165 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf9d11d-573b-4cca-9e5d-26e244088816 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:02.151443 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf9d11d-573b-4cca-9e5d-26e244088816 None admin] PATH_INFO: `/v3/services` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:02.152223 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf9d11d-573b-4cca-9e5d-26e244088816 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:02.152578 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf9d11d-573b-4cca-9e5d-26e244088816 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:02.154214 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf9d11d-573b-4cca-9e5d-26e244088816 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:02.172662 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 369/764] 10.222.0.20 () {56 vars in 1124 bytes} [Tue Sep 6 20:00:01 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 945 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 20:00:02.184635 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf9d11d-573b-4cca-9e5d-26e244088816 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:02.307475 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:02.357540 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c20c23d-e74f-44f1-bc7f-ae44321654bb None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:02.391393 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf9d11d-573b-4cca-9e5d-26e244088816 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:02.400259 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83305d1-d290-4676-a0d8-0af5fd2ed2a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:02.400890 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83305d1-d290-4676-a0d8-0af5fd2ed2a6 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:02.401219 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83305d1-d290-4676-a0d8-0af5fd2ed2a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:02.401504 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83305d1-d290-4676-a0d8-0af5fd2ed2a6 None admin] PATH_INFO: `/v3/users` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:02.402303 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83305d1-d290-4676-a0d8-0af5fd2ed2a6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:02.402616 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83305d1-d290-4676-a0d8-0af5fd2ed2a6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:02.404279 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83305d1-d290-4676-a0d8-0af5fd2ed2a6 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:02.428064 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 396/765] 10.222.0.20 () {56 vars in 1133 bytes} [Tue Sep 6 20:00:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 20:00:02.429583 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61212b1-9153-4566-ae4a-4c874c80109c None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:02.429948 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61212b1-9153-4566-ae4a-4c874c80109c None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:02.430235 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61212b1-9153-4566-ae4a-4c874c80109c None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:02.432036 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 397/766] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:00:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:00:02.433504 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55fc663-c10a-4e86-8337-5a7a787dc372 None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:02.433941 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55fc663-c10a-4e86-8337-5a7a787dc372 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:02.434222 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55fc663-c10a-4e86-8337-5a7a787dc372 None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:02.435906 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 398/767] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:00:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:00:02.442466 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f95144-a8b3-4286-8f07-281f20be348d None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:02.442768 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f95144-a8b3-4286-8f07-281f20be348d None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:02.443050 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f95144-a8b3-4286-8f07-281f20be348d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:02.463462 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f40eca7-f0a0-4a35-8825-7f6ce6dc4948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:02.464331 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-08f95144-a8b3-4286-8f07-281f20be348d None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:00:02.465254 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f40eca7-f0a0-4a35-8825-7f6ce6dc4948 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:02.465635 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-08f95144-a8b3-4286-8f07-281f20be348d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:00:02.466200 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f40eca7-f0a0-4a35-8825-7f6ce6dc4948 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:02.466991 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f40eca7-f0a0-4a35-8825-7f6ce6dc4948 None admin] PATH_INFO: `/v3/services` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:02.468430 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f40eca7-f0a0-4a35-8825-7f6ce6dc4948 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:02.469204 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f40eca7-f0a0-4a35-8825-7f6ce6dc4948 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:02.471777 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f40eca7-f0a0-4a35-8825-7f6ce6dc4948 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:02.489164 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 370/768] 10.222.0.20 () {56 vars in 1124 bytes} [Tue Sep 6 20:00:02 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:00:02.489164 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f40eca7-f0a0-4a35-8825-7f6ce6dc4948 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:02.525444 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:02.569138 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 399/769] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:00:02 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 20:00:02.571552 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fcc987-1532-4bb6-b441-a96ec26a6d3a None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:02.572062 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fcc987-1532-4bb6-b441-a96ec26a6d3a None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:02.572548 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fcc987-1532-4bb6-b441-a96ec26a6d3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:02.600046 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-66fcc987-1532-4bb6-b441-a96ec26a6d3a None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:00:02.601746 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-66fcc987-1532-4bb6-b441-a96ec26a6d3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:00:02.691987 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f40eca7-f0a0-4a35-8825-7f6ce6dc4948 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:02.728014 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 400/770] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:00:02 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 20:00:02.728014 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fcc987-1532-4bb6-b441-a96ec26a6d3a None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:02.762465 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:02.782771 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa95684a-e893-4d2b-9901-33247c3dc174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:02.783842 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa95684a-e893-4d2b-9901-33247c3dc174 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:02.784348 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa95684a-e893-4d2b-9901-33247c3dc174 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:02.784836 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa95684a-e893-4d2b-9901-33247c3dc174 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:02.806435 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa95684a-e893-4d2b-9901-33247c3dc174 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:02.809202 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa95684a-e893-4d2b-9901-33247c3dc174 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:02.840204 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-aa95684a-e893-4d2b-9901-33247c3dc174 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 20:00:02.840204 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:02.840204 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:00:02.840204 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:00:02.840204 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:00:02.840204 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:00:02.840204 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:00:02.840204 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:00:02.840204 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:00:02.840204 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:00:02.840204 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:00:02.840204 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:00:02.840204 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 20:00:02.840204 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 20:00:02.840204 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 20:00:02.840204 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 20:00:02.841359 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:00:02.841359 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:00:02.841359 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:00:02.841359 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:00:02.841359 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:00:02.841359 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:00:02.841359 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:00:02.841359 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 20:00:02.841359 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:00:02.841359 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:00:02.841359 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:00:02.841359 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:00:02.841359 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:00:02.841359 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:00:02.841359 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:00:02.841359 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:00:02.841359 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 20:00:02.842381 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 20:00:02.842381 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 20:00:02.842381 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:00:02.849821 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 371/771] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 20:00:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 358 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:00:02.852360 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b502049-2684-44ac-a4dd-010c902ed892 None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:02.853219 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b502049-2684-44ac-a4dd-010c902ed892 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:02.853759 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b502049-2684-44ac-a4dd-010c902ed892 None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:02.857795 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 372/772] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:00:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:00:02.859510 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b502049-2684-44ac-a4dd-010c902ed892 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:02.914835 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fcc987-1532-4bb6-b441-a96ec26a6d3a None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:02.918944 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:02.975146 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26e7faa-2616-4b40-ba5b-4a73b2ccbd8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:02.976833 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26e7faa-2616-4b40-ba5b-4a73b2ccbd8b None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:02.977366 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26e7faa-2616-4b40-ba5b-4a73b2ccbd8b None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:02.977895 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26e7faa-2616-4b40-ba5b-4a73b2ccbd8b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:02.979214 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26e7faa-2616-4b40-ba5b-4a73b2ccbd8b None admin] RBAC: Adding query filter params (interface=public, service_id=d8eff2f98eb3453bb3016d10b68c4105, region_id=RegionOne) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:02.979754 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26e7faa-2616-4b40-ba5b-4a73b2ccbd8b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:02.981822 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26e7faa-2616-4b40-ba5b-4a73b2ccbd8b None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:02.998153 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 401/773] 10.222.0.20 () {56 vars in 1251 bytes} [Tue Sep 6 20:00:02 2022] GET /identity/v3/endpoints?service_id=d8eff2f98eb3453bb3016d10b68c4105&interface=public®ion_id=RegionOne => generated 194 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:00:02.998153 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b184bbb4-f065-4800-ac7e-47f503b95e04 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:02.998153 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b184bbb4-f065-4800-ac7e-47f503b95e04 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:02.998153 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b184bbb4-f065-4800-ac7e-47f503b95e04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:03.026860 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-b184bbb4-f065-4800-ac7e-47f503b95e04 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:00:03.028741 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-b184bbb4-f065-4800-ac7e-47f503b95e04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:00:03.070769 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b502049-2684-44ac-a4dd-010c902ed892 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:03.113657 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14629774-87a6-41ae-b1ce-58a5f6c8c7f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:03.114207 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14629774-87a6-41ae-b1ce-58a5f6c8c7f0 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:03.114529 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14629774-87a6-41ae-b1ce-58a5f6c8c7f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:03.114849 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14629774-87a6-41ae-b1ce-58a5f6c8c7f0 None admin] PATH_INFO: `/v3/domains` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:03.115524 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14629774-87a6-41ae-b1ce-58a5f6c8c7f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:03.115829 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14629774-87a6-41ae-b1ce-58a5f6c8c7f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:03.117251 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14629774-87a6-41ae-b1ce-58a5f6c8c7f0 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:03.137705 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 373/774] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 6 20:00:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:00:03.137705 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14629774-87a6-41ae-b1ce-58a5f6c8c7f0 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:03.138105 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 402/775] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:00:02 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 20:00:03.138791 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b184bbb4-f065-4800-ac7e-47f503b95e04 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:03.187766 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:03.281390 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14629774-87a6-41ae-b1ce-58a5f6c8c7f0 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:03.292525 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:03.298852 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b184bbb4-f065-4800-ac7e-47f503b95e04 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:03.333071 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11365e8d-5fca-41f7-b41c-e6a630f543cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:03.333753 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11365e8d-5fca-41f7-b41c-e6a630f543cf None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:03.334195 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11365e8d-5fca-41f7-b41c-e6a630f543cf None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:03.334474 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11365e8d-5fca-41f7-b41c-e6a630f543cf None admin] PATH_INFO: `/v3/services` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:03.335113 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11365e8d-5fca-41f7-b41c-e6a630f543cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:03.336573 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11365e8d-5fca-41f7-b41c-e6a630f543cf None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:03.350953 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 374/776] 10.222.0.20 () {56 vars in 1087 bytes} [Tue Sep 6 20:00:03 2022] GET /identity/v3/services => generated 772 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:00:03.351911 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11365e8d-5fca-41f7-b41c-e6a630f543cf None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:03.361032 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c879b5-31d6-4448-863e-6acf6addf833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:03.362657 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c879b5-31d6-4448-863e-6acf6addf833 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:03.363400 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c879b5-31d6-4448-863e-6acf6addf833 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:03.364203 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c879b5-31d6-4448-863e-6acf6addf833 None admin] PATH_INFO: `/v3/services/network` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:03.366759 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c879b5-31d6-4448-863e-6acf6addf833 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:03.370596 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c879b5-31d6-4448-863e-6acf6addf833 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:03.396597 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-63c879b5-31d6-4448-863e-6acf6addf833 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 06 20:00:03.396597 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:03.396597 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 20:00:03.396597 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 20:00:03.396597 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 20:00:03.396597 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 20:00:03.396597 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 20:00:03.396597 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 20:00:03.396597 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 06 20:00:03.396597 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 20:00:03.396597 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 20:00:03.396597 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 20:00:03.396597 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:03.396597 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:00:03.396597 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:00:03.396597 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:00:03.396597 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:00:03.398557 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:00:03.398557 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:00:03.398557 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:00:03.398557 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:00:03.398557 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:00:03.398557 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:00:03.398557 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 20:00:03.398557 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 20:00:03.398557 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 20:00:03.398557 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 20:00:03.398557 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:00:03.398557 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:00:03.398557 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:00:03.398557 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:00:03.398557 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:00:03.398557 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:00:03.400274 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:00:03.400274 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 20:00:03.400274 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:00:03.400274 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:00:03.400274 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:00:03.400274 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:00:03.400274 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:00:03.400274 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:00:03.400274 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:00:03.400274 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:00:03.400274 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 20:00:03.400274 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 20:00:03.400274 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 06 20:00:03.400274 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 20:00:03.405304 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 403/777] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 6 20:00:03 2022] GET /identity/v3/services/network => generated 88 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:00:03.407386 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c879b5-31d6-4448-863e-6acf6addf833 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:03.559717 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:03.569416 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11365e8d-5fca-41f7-b41c-e6a630f543cf None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:03.609391 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c879b5-31d6-4448-863e-6acf6addf833 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:03.638517 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968f952b-b476-486c-8f2c-335d0512c093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:03.640614 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968f952b-b476-486c-8f2c-335d0512c093 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:03.641136 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968f952b-b476-486c-8f2c-335d0512c093 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:03.641672 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968f952b-b476-486c-8f2c-335d0512c093 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:03.657833 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968f952b-b476-486c-8f2c-335d0512c093 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:03.662120 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968f952b-b476-486c-8f2c-335d0512c093 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:03.688429 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-968f952b-b476-486c-8f2c-335d0512c093 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 20:00:03.688429 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:03.688429 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:00:03.688429 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:00:03.688429 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:00:03.688429 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:00:03.688429 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:00:03.688429 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:00:03.688429 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:00:03.688429 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:00:03.688429 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:00:03.688429 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:00:03.688429 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 20:00:03.688429 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 20:00:03.688429 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 20:00:03.689511 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 20:00:03.689511 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:00:03.689511 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:00:03.689511 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:00:03.689511 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:00:03.689511 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:00:03.689511 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:00:03.689511 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:00:03.689511 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 20:00:03.689511 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:00:03.689511 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:00:03.689511 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:00:03.689511 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:00:03.689511 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:00:03.689511 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:00:03.689511 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:00:03.689511 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:00:03.690693 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 20:00:03.690693 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 20:00:03.690693 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 20:00:03.690693 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 20:00:03.690693 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 20:00:03.690693 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 20:00:03.690693 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 20:00:03.690693 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:00:03.691199 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa84874-2216-4364-a3f4-39842fd25f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:03.691199 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa84874-2216-4364-a3f4-39842fd25f96 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:03.691199 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa84874-2216-4364-a3f4-39842fd25f96 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:03.691410 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 375/778] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 6 20:00:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:00:03.692531 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968f952b-b476-486c-8f2c-335d0512c093 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:03.693241 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa84874-2216-4364-a3f4-39842fd25f96 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:03.712485 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa84874-2216-4364-a3f4-39842fd25f96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 20:00:03.720820 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-aaa84874-2216-4364-a3f4-39842fd25f96 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 20:00:03.720820 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:03.720820 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:00:03.720820 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:00:03.720820 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:00:03.720820 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:00:03.720820 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:00:03.720820 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:00:03.720820 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:00:03.720820 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:00:03.720820 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:00:03.720820 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:00:03.720820 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 20:00:03.720820 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 20:00:03.720820 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 20:00:03.720820 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 20:00:03.723070 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 20:00:03.723070 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 20:00:03.723070 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 20:00:03.723070 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 20:00:03.723070 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:00:03.723070 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:00:03.723070 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:00:03.723070 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:00:03.723070 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:00:03.723070 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:00:03.723070 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:00:03.723070 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 20:00:03.723070 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:00:03.723070 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:00:03.723070 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:00:03.723070 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:00:03.725236 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:00:03.725236 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:00:03.725236 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:00:03.725236 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:00:03.725236 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 20:00:03.725236 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 20:00:03.725236 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 20:00:03.725236 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 20:00:03.725236 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 20:00:03.725236 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 20:00:03.725236 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 20:00:03.725236 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 20:00:03.725236 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 404/779] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Sep 6 20:00:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:00:03.725236 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa84874-2216-4364-a3f4-39842fd25f96 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:03.759477 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:03.867711 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968f952b-b476-486c-8f2c-335d0512c093 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:03.883724 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa84874-2216-4364-a3f4-39842fd25f96 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:03.930217 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991a3362-fcf7-48ff-972e-bb12ac4eac15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:03.931146 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991a3362-fcf7-48ff-972e-bb12ac4eac15 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:03.931658 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991a3362-fcf7-48ff-972e-bb12ac4eac15 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:03.932398 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991a3362-fcf7-48ff-972e-bb12ac4eac15 None admin] PATH_INFO: `/v3/services` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:03.933681 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991a3362-fcf7-48ff-972e-bb12ac4eac15 None admin] RBAC: Adding query filter params (name=network) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:03.934174 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991a3362-fcf7-48ff-972e-bb12ac4eac15 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:03.936800 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991a3362-fcf7-48ff-972e-bb12ac4eac15 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:03.951880 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 376/780] 10.222.0.20 () {56 vars in 1112 bytes} [Tue Sep 6 20:00:03 2022] GET /identity/v3/services?name=network => generated 124 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:00:03.951880 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991a3362-fcf7-48ff-972e-bb12ac4eac15 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:03.954395 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba2ad8b-2e15-485f-994a-123251f8cadf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:03.955461 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba2ad8b-2e15-485f-994a-123251f8cadf None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:03.956081 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba2ad8b-2e15-485f-994a-123251f8cadf None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:03.956590 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba2ad8b-2e15-485f-994a-123251f8cadf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:03.980854 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba2ad8b-2e15-485f-994a-123251f8cadf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 20:00:03.988565 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-dba2ad8b-2e15-485f-994a-123251f8cadf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 20:00:03.988565 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:03.988565 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:00:03.988565 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:00:03.988565 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:00:03.988565 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:00:03.988565 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:00:03.988565 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:00:03.988565 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:00:03.988565 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:00:03.988565 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:00:03.988565 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:00:03.988565 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 20:00:03.988565 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 20:00:03.988565 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 20:00:03.988565 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 20:00:03.994206 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 20:00:03.994206 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 20:00:03.994206 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 20:00:03.994206 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 20:00:03.994206 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:00:03.994206 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:00:03.994206 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:00:03.994206 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:00:03.994206 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:00:03.994206 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:00:03.994206 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:00:03.994206 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 20:00:03.994206 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:00:03.994206 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:00:03.994206 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:00:03.994206 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:00:03.996045 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:00:03.996045 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:00:03.996045 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:00:03.996045 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:00:03.996045 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 20:00:03.996045 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 20:00:03.996045 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 20:00:03.996045 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 20:00:03.996045 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 20:00:03.996045 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 20:00:03.996045 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 20:00:03.996045 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 20:00:03.996045 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 405/781] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Sep 6 20:00:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:00:03.996045 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba2ad8b-2e15-485f-994a-123251f8cadf None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:04.004303 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:04.124041 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:04.153077 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba2ad8b-2e15-485f-994a-123251f8cadf None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:04.170933 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991a3362-fcf7-48ff-972e-bb12ac4eac15 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:04.203294 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d346a4b2-ccf3-4abb-9c17-9107592cf766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:04.203885 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d346a4b2-ccf3-4abb-9c17-9107592cf766 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:04.204274 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d346a4b2-ccf3-4abb-9c17-9107592cf766 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:04.204556 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d346a4b2-ccf3-4abb-9c17-9107592cf766 None admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:04.205449 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d346a4b2-ccf3-4abb-9c17-9107592cf766 None admin] RBAC: Adding query filter params (name=service) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:04.205727 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d346a4b2-ccf3-4abb-9c17-9107592cf766 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:04.207261 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d346a4b2-ccf3-4abb-9c17-9107592cf766 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:04.223730 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 406/782] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Sep 6 20:00:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:00:04.223730 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d346a4b2-ccf3-4abb-9c17-9107592cf766 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:04.248722 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32283d9a-a534-4bdd-beef-e023656d710d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:04.249950 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32283d9a-a534-4bdd-beef-e023656d710d None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:04.250329 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32283d9a-a534-4bdd-beef-e023656d710d None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:04.250718 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32283d9a-a534-4bdd-beef-e023656d710d None admin] PATH_INFO: `/v3/projects` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:04.251784 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32283d9a-a534-4bdd-beef-e023656d710d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:04.252235 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32283d9a-a534-4bdd-beef-e023656d710d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:04.254386 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32283d9a-a534-4bdd-beef-e023656d710d None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:04.280046 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 377/783] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Sep 6 20:00:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:00:04.280046 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32283d9a-a534-4bdd-beef-e023656d710d None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:04.407422 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d346a4b2-ccf3-4abb-9c17-9107592cf766 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:04.446145 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:04.466216 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80be626-9545-425f-92f5-b56371519841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:04.466892 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80be626-9545-425f-92f5-b56371519841 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:04.467708 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80be626-9545-425f-92f5-b56371519841 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:04.467708 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80be626-9545-425f-92f5-b56371519841 None admin] PATH_INFO: `/v3/services` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:04.468945 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80be626-9545-425f-92f5-b56371519841 None admin] RBAC: Adding query filter params (type=network) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:04.469103 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80be626-9545-425f-92f5-b56371519841 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:04.470659 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80be626-9545-425f-92f5-b56371519841 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:04.486030 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 407/784] 10.222.0.20 () {56 vars in 1112 bytes} [Tue Sep 6 20:00:04 2022] GET /identity/v3/services?type=network => generated 350 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:00:04.486030 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80be626-9545-425f-92f5-b56371519841 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:04.486455 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32283d9a-a534-4bdd-beef-e023656d710d None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:04.503151 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:04.550290 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690f57bc-d9c5-4179-a8c6-e2757bfca979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:04.551340 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690f57bc-d9c5-4179-a8c6-e2757bfca979 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:04.551828 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690f57bc-d9c5-4179-a8c6-e2757bfca979 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:04.552292 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690f57bc-d9c5-4179-a8c6-e2757bfca979 None admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:04.553555 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690f57bc-d9c5-4179-a8c6-e2757bfca979 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:04.554127 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690f57bc-d9c5-4179-a8c6-e2757bfca979 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:04.556788 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690f57bc-d9c5-4179-a8c6-e2757bfca979 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:04.576509 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 378/785] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 6 20:00:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:00:04.578670 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690f57bc-d9c5-4179-a8c6-e2757bfca979 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:04.663695 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80be626-9545-425f-92f5-b56371519841 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:04.718047 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58c67d0-8bf5-4b53-98c8-a8380ec4e76d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:04.718769 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58c67d0-8bf5-4b53-98c8-a8380ec4e76d None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:04.719055 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58c67d0-8bf5-4b53-98c8-a8380ec4e76d None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:04.719459 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58c67d0-8bf5-4b53-98c8-a8380ec4e76d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:04.736536 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58c67d0-8bf5-4b53-98c8-a8380ec4e76d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:04.738118 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58c67d0-8bf5-4b53-98c8-a8380ec4e76d None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:04.768048 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-e58c67d0-8bf5-4b53-98c8-a8380ec4e76d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 20:00:04.768048 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:04.768048 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:00:04.768048 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:00:04.768048 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:00:04.768048 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:00:04.768048 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:00:04.768048 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:00:04.768048 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:00:04.768048 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:00:04.768048 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:00:04.768048 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:00:04.768048 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 20:00:04.768048 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 20:00:04.768048 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 20:00:04.768048 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 20:00:04.769783 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:00:04.769783 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:00:04.769783 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:00:04.769783 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:00:04.769783 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:00:04.769783 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:00:04.769783 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:00:04.769783 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 20:00:04.769783 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:00:04.769783 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:00:04.769783 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:00:04.769783 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:00:04.769783 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:00:04.769783 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:00:04.769783 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:00:04.769783 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:00:04.769783 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 20:00:04.770865 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 20:00:04.770865 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 20:00:04.770865 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 20:00:04.774046 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 408/786] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 20:00:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:00:04.775366 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58c67d0-8bf5-4b53-98c8-a8380ec4e76d None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:04.803674 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690f57bc-d9c5-4179-a8c6-e2757bfca979 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:04.875674 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc1709c-82c6-415d-9610-64e8279069d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:04.876500 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc1709c-82c6-415d-9610-64e8279069d1 None admin] REQUEST_METHOD: `PUT` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:04.876812 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc1709c-82c6-415d-9610-64e8279069d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:04.877289 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc1709c-82c6-415d-9610-64e8279069d1 None admin] PATH_INFO: `/v3/projects/6aef43493bb1420a9e122313d8db7d5a/users/5776cbf4f6714b2083c3ebdd6d98cc87/roles/e0ab41d8ae22474688b5361bcc75e0f0` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:04.879532 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc1709c-82c6-415d-9610-64e8279069d1 None admin] RBAC: Authorizing `identity:create_grant(project_id=6aef43493bb1420a9e122313d8db7d5a, user_id=5776cbf4f6714b2083c3ebdd6d98cc87, role_id=e0ab41d8ae22474688b5361bcc75e0f0)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:04.882553 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc1709c-82c6-415d-9610-64e8279069d1 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:04.905084 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 379/787] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Sep 6 20:00:04 2022] PUT /identity/v3/projects/6aef43493bb1420a9e122313d8db7d5a/users/5776cbf4f6714b2083c3ebdd6d98cc87/roles/e0ab41d8ae22474688b5361bcc75e0f0 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:00:04.905386 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:04.916990 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc1709c-82c6-415d-9610-64e8279069d1 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:04.938751 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:04.991669 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58c67d0-8bf5-4b53-98c8-a8380ec4e76d None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:05.051570 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dff361-d193-47c2-a2c7-230666b8bcab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:05.052532 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dff361-d193-47c2-a2c7-230666b8bcab None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:05.053012 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dff361-d193-47c2-a2c7-230666b8bcab None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:05.053454 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dff361-d193-47c2-a2c7-230666b8bcab None admin] PATH_INFO: `/v3/endpoints` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:05.054548 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dff361-d193-47c2-a2c7-230666b8bcab None admin] RBAC: Adding query filter params (interface=public, service_id=fec009ed90074dcba7f00b9e5caa9ce1, region_id=RegionOne) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:05.055067 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dff361-d193-47c2-a2c7-230666b8bcab None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:05.056865 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dff361-d193-47c2-a2c7-230666b8bcab None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:05.064665 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 409/788] 10.222.0.20 () {56 vars in 1251 bytes} [Tue Sep 6 20:00:04 2022] GET /identity/v3/endpoints?service_id=fec009ed90074dcba7f00b9e5caa9ce1&interface=public®ion_id=RegionOne => generated 194 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:00:05.066042 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dff361-d193-47c2-a2c7-230666b8bcab None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:05.171139 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc1709c-82c6-415d-9610-64e8279069d1 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:05.226288 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1904ac06-68d9-42c6-93ba-09c9db1a3a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:05.227459 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1904ac06-68d9-42c6-93ba-09c9db1a3a37 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:05.228066 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1904ac06-68d9-42c6-93ba-09c9db1a3a37 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:05.228678 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1904ac06-68d9-42c6-93ba-09c9db1a3a37 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:05.251969 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1904ac06-68d9-42c6-93ba-09c9db1a3a37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:05.256446 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1904ac06-68d9-42c6-93ba-09c9db1a3a37 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:05.276490 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-1904ac06-68d9-42c6-93ba-09c9db1a3a37 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 20:00:05.276490 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:05.276490 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:00:05.276490 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:00:05.276490 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:00:05.276490 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:00:05.276490 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:00:05.276490 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:00:05.276490 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:00:05.276490 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:00:05.276490 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:00:05.276490 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:00:05.276490 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 20:00:05.276490 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 20:00:05.276490 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 20:00:05.276490 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 20:00:05.277831 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:00:05.277831 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:00:05.277831 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:00:05.277831 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:00:05.277831 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:00:05.277831 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:00:05.277831 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:00:05.277831 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 20:00:05.277831 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:00:05.277831 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:00:05.277831 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:00:05.277831 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:00:05.277831 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:00:05.277831 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:00:05.277831 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:00:05.277831 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:00:05.277831 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 20:00:05.279141 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 20:00:05.279141 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 20:00:05.279141 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:00:05.281872 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 380/789] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 20:00:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 376 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:00:05.283071 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1904ac06-68d9-42c6-93ba-09c9db1a3a37 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:05.305528 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dff361-d193-47c2-a2c7-230666b8bcab None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:05.354212 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:05.370015 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:05.377867 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36702e27-f4e9-4508-b2bb-2214a3a959a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:05.379095 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36702e27-f4e9-4508-b2bb-2214a3a959a3 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:05.379645 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36702e27-f4e9-4508-b2bb-2214a3a959a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:05.380206 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36702e27-f4e9-4508-b2bb-2214a3a959a3 None admin] PATH_INFO: `/v3/domains` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:05.381545 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36702e27-f4e9-4508-b2bb-2214a3a959a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:05.382236 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36702e27-f4e9-4508-b2bb-2214a3a959a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:05.385011 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36702e27-f4e9-4508-b2bb-2214a3a959a3 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:05.411556 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 410/790] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 6 20:00:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:00:05.413370 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36702e27-f4e9-4508-b2bb-2214a3a959a3 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:05.433483 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1904ac06-68d9-42c6-93ba-09c9db1a3a37 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:05.483097 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be009c9-7f0f-4615-bc68-69124b183c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:05.484618 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be009c9-7f0f-4615-bc68-69124b183c6a None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:05.485141 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be009c9-7f0f-4615-bc68-69124b183c6a None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:05.485600 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be009c9-7f0f-4615-bc68-69124b183c6a None admin] PATH_INFO: `/v3/services` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:05.486471 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be009c9-7f0f-4615-bc68-69124b183c6a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:05.488185 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be009c9-7f0f-4615-bc68-69124b183c6a None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:05.495824 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 381/791] 10.222.0.20 () {56 vars in 1087 bytes} [Tue Sep 6 20:00:05 2022] GET /identity/v3/services => generated 772 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:00:05.497290 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be009c9-7f0f-4615-bc68-69124b183c6a None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:05.674118 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36702e27-f4e9-4508-b2bb-2214a3a959a3 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:05.707098 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be009c9-7f0f-4615-bc68-69124b183c6a None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:05.749143 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:05.779515 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5087ec01-e5e7-4909-bf9d-348e9d5b6965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:05.780252 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5087ec01-e5e7-4909-bf9d-348e9d5b6965 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:05.780677 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5087ec01-e5e7-4909-bf9d-348e9d5b6965 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:05.781023 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5087ec01-e5e7-4909-bf9d-348e9d5b6965 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:05.782688 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16d9c3d-2f2f-4048-92c2-c5335c58c53f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:05.783331 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16d9c3d-2f2f-4048-92c2-c5335c58c53f None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:05.783612 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16d9c3d-2f2f-4048-92c2-c5335c58c53f None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:05.783973 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16d9c3d-2f2f-4048-92c2-c5335c58c53f None admin] PATH_INFO: `/v3/domains` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:05.784849 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16d9c3d-2f2f-4048-92c2-c5335c58c53f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:05.785194 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16d9c3d-2f2f-4048-92c2-c5335c58c53f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:05.787086 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16d9c3d-2f2f-4048-92c2-c5335c58c53f None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:05.798777 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5087ec01-e5e7-4909-bf9d-348e9d5b6965 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:05.800446 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5087ec01-e5e7-4909-bf9d-348e9d5b6965 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:05.816506 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-5087ec01-e5e7-4909-bf9d-348e9d5b6965 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 06 20:00:05.816506 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:05.816506 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:00:05.816506 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:00:05.816506 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:00:05.816506 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:00:05.816506 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:00:05.816506 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:00:05.816506 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:00:05.816506 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:00:05.816506 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:00:05.816506 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:00:05.816506 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 20:00:05.816506 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 20:00:05.816506 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 20:00:05.816506 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 20:00:05.817538 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:00:05.817538 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:00:05.817538 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 20:00:05.817538 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 20:00:05.817538 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 20:00:05.817538 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 20:00:05.817538 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:00:05.817538 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:00:05.817538 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:00:05.817538 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:00:05.817538 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:00:05.817538 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 20:00:05.817538 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:00:05.817538 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:00:05.817538 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:00:05.817538 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:00:05.817538 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:00:05.817538 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:00:05.819290 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:00:05.819290 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:00:05.819290 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 20:00:05.819290 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 20:00:05.819290 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 20:00:05.819290 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 20:00:05.819290 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 20:00:05.819290 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 20:00:05.819290 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 06 20:00:05.819290 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:00:05.819290 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 382/792] 10.222.0.20 () {56 vars in 1117 bytes} [Tue Sep 6 20:00:05 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:00:05.825512 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5087ec01-e5e7-4909-bf9d-348e9d5b6965 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:05.831046 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 411/793] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 6 20:00:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 419 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:00:05.837355 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16d9c3d-2f2f-4048-92c2-c5335c58c53f None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:05.896023 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:06.000061 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16d9c3d-2f2f-4048-92c2-c5335c58c53f None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:06.032111 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5087ec01-e5e7-4909-bf9d-348e9d5b6965 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:06.053213 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec39ddfa-86ec-448a-9377-5749410b0dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:06.054039 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec39ddfa-86ec-448a-9377-5749410b0dfe None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:06.054362 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec39ddfa-86ec-448a-9377-5749410b0dfe None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:06.054955 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec39ddfa-86ec-448a-9377-5749410b0dfe None admin] PATH_INFO: `/v3/users/swift` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:06.066332 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec39ddfa-86ec-448a-9377-5749410b0dfe None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:06.068196 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec39ddfa-86ec-448a-9377-5749410b0dfe None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:06.082436 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-ec39ddfa-86ec-448a-9377-5749410b0dfe None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 06 20:00:06.082436 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:06.082436 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:00:06.082436 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:00:06.082436 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:00:06.082436 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:00:06.082436 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:00:06.082436 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:00:06.082436 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:00:06.082436 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:00:06.082436 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:00:06.082436 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:00:06.082436 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 20:00:06.082436 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 20:00:06.082436 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 20:00:06.082436 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 20:00:06.084101 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:00:06.084101 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:00:06.084101 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 20:00:06.084101 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 20:00:06.084101 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 20:00:06.084101 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 20:00:06.084101 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:00:06.084101 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:00:06.084101 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:00:06.084101 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:00:06.084101 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:00:06.084101 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 20:00:06.084101 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:00:06.084101 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:00:06.084101 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:00:06.084101 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:00:06.084101 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:00:06.084101 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:00:06.085810 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:00:06.085810 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:00:06.085810 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 20:00:06.085810 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 20:00:06.085810 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 20:00:06.085810 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 20:00:06.085810 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 20:00:06.085810 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 20:00:06.085810 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 06 20:00:06.085810 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 20:00:06.085810 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 412/794] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Sep 6 20:00:05 2022] GET /identity/v3/users/swift => generated 83 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:00:06.096179 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec39ddfa-86ec-448a-9377-5749410b0dfe None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:06.098438 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196a4e38-9971-4c87-8cd3-a08f462d2088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:06.099079 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196a4e38-9971-4c87-8cd3-a08f462d2088 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:06.099473 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196a4e38-9971-4c87-8cd3-a08f462d2088 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:06.099750 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196a4e38-9971-4c87-8cd3-a08f462d2088 None admin] PATH_INFO: `/v3/users` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:06.103803 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196a4e38-9971-4c87-8cd3-a08f462d2088 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:06.104216 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196a4e38-9971-4c87-8cd3-a08f462d2088 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:06.105738 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196a4e38-9971-4c87-8cd3-a08f462d2088 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:06.147139 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 383/795] 10.222.0.20 () {56 vars in 1149 bytes} [Tue Sep 6 20:00:05 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:00:06.154066 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196a4e38-9971-4c87-8cd3-a08f462d2088 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:06.231015 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:06.308957 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196a4e38-9971-4c87-8cd3-a08f462d2088 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:06.334000 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec39ddfa-86ec-448a-9377-5749410b0dfe None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:06.339469 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:06.358169 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65861b71-bafa-4a8b-ab88-37d080d904bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:06.359531 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65861b71-bafa-4a8b-ab88-37d080d904bd None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:06.359810 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65861b71-bafa-4a8b-ab88-37d080d904bd None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:06.360084 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65861b71-bafa-4a8b-ab88-37d080d904bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:06.382085 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65861b71-bafa-4a8b-ab88-37d080d904bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:06.385118 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65861b71-bafa-4a8b-ab88-37d080d904bd None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:06.396354 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6a113a-d2e8-4cbd-a4b2-da234d5a5dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:06.397008 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6a113a-d2e8-4cbd-a4b2-da234d5a5dab None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:06.397286 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6a113a-d2e8-4cbd-a4b2-da234d5a5dab None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:06.397562 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6a113a-d2e8-4cbd-a4b2-da234d5a5dab None admin] PATH_INFO: `/v3/users` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:06.398365 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6a113a-d2e8-4cbd-a4b2-da234d5a5dab None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:06.398694 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6a113a-d2e8-4cbd-a4b2-da234d5a5dab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:06.400461 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6a113a-d2e8-4cbd-a4b2-da234d5a5dab None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:06.419719 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-65861b71-bafa-4a8b-ab88-37d080d904bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 20:00:06.419719 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:06.419719 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:00:06.419719 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:00:06.419719 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:00:06.419719 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:00:06.419719 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:00:06.419719 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:00:06.419719 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:00:06.419719 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:00:06.419719 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:00:06.419719 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:00:06.419719 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 20:00:06.419719 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 20:00:06.419719 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 20:00:06.419719 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 20:00:06.422135 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:00:06.422135 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:00:06.422135 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:00:06.422135 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:00:06.422135 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:00:06.422135 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:00:06.422135 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:00:06.422135 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 20:00:06.422135 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:00:06.422135 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:00:06.422135 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:00:06.422135 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:00:06.422135 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:00:06.422135 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:00:06.422135 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:00:06.422135 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:00:06.422135 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 20:00:06.423965 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 20:00:06.423965 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 20:00:06.423965 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:00:06.427606 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 413/797] 10.222.0.20 () {56 vars in 1135 bytes} [Tue Sep 6 20:00:06 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:00:06.427991 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 384/797] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 20:00:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:00:06.433652 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00c48dd-bd34-424c-8df9-3ea6ca209502 None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:06.433652 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00c48dd-bd34-424c-8df9-3ea6ca209502 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:06.433652 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00c48dd-bd34-424c-8df9-3ea6ca209502 None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:06.434011 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65861b71-bafa-4a8b-ab88-37d080d904bd None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:06.434197 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 414/798] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:00:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:00:06.445523 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00c48dd-bd34-424c-8df9-3ea6ca209502 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:06.615109 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00c48dd-bd34-424c-8df9-3ea6ca209502 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:06.671680 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41aa30d-4fd9-47f3-a504-e51e81bec56a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:06.672417 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41aa30d-4fd9-47f3-a504-e51e81bec56a None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:06.672700 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41aa30d-4fd9-47f3-a504-e51e81bec56a None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:06.672977 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41aa30d-4fd9-47f3-a504-e51e81bec56a None admin] PATH_INFO: `/v3/domains` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:06.673710 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41aa30d-4fd9-47f3-a504-e51e81bec56a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:06.674019 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41aa30d-4fd9-47f3-a504-e51e81bec56a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:06.675507 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41aa30d-4fd9-47f3-a504-e51e81bec56a None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:06.697658 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 415/799] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 6 20:00:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 20:00:06.697658 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f624a9-2660-44e9-9c96-606c4b4d6382 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:06.697658 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f624a9-2660-44e9-9c96-606c4b4d6382 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:06.697658 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f624a9-2660-44e9-9c96-606c4b4d6382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:06.705334 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:06.719217 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-61f624a9-2660-44e9-9c96-606c4b4d6382 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:00:06.720593 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-61f624a9-2660-44e9-9c96-606c4b4d6382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:00:06.728637 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65861b71-bafa-4a8b-ab88-37d080d904bd None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:06.743667 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:06.821181 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a681c2c0-02e1-4680-b68e-c485b1186681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:06.822720 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a681c2c0-02e1-4680-b68e-c485b1186681 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:06.823443 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a681c2c0-02e1-4680-b68e-c485b1186681 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:06.824110 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a681c2c0-02e1-4680-b68e-c485b1186681 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:06.831424 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 416/800] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:00:06 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 20:00:06.831424 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f624a9-2660-44e9-9c96-606c4b4d6382 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:06.853073 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a681c2c0-02e1-4680-b68e-c485b1186681 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:06.855526 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a681c2c0-02e1-4680-b68e-c485b1186681 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:06.887764 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-a681c2c0-02e1-4680-b68e-c485b1186681 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 20:00:06.887764 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:06.887764 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:00:06.887764 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:00:06.887764 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:00:06.887764 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:00:06.887764 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:00:06.887764 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:00:06.887764 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:00:06.887764 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:00:06.887764 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:00:06.887764 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:00:06.887764 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 20:00:06.887764 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 20:00:06.887764 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 20:00:06.887764 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 20:00:06.890002 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:00:06.890002 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:00:06.890002 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:00:06.890002 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:00:06.890002 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:00:06.890002 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:00:06.890002 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:00:06.890002 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 20:00:06.890002 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:00:06.890002 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:00:06.890002 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:00:06.890002 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:00:06.890002 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:00:06.890002 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:00:06.890002 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:00:06.890002 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:00:06.890002 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 20:00:06.891807 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 20:00:06.891807 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 20:00:06.891807 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:00:06.896449 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 385/801] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 20:00:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 460 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:00:06.902361 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a681c2c0-02e1-4680-b68e-c485b1186681 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:07.082115 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f624a9-2660-44e9-9c96-606c4b4d6382 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:07.149389 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a681c2c0-02e1-4680-b68e-c485b1186681 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:07.155815 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7462d0bb-5175-49ae-b1b7-68ad59f20041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:07.156348 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7462d0bb-5175-49ae-b1b7-68ad59f20041 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:07.156647 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7462d0bb-5175-49ae-b1b7-68ad59f20041 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:07.157006 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7462d0bb-5175-49ae-b1b7-68ad59f20041 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:07.174618 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7462d0bb-5175-49ae-b1b7-68ad59f20041 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:07.181666 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7462d0bb-5175-49ae-b1b7-68ad59f20041 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:07.192423 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-7462d0bb-5175-49ae-b1b7-68ad59f20041 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 20:00:07.192423 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:07.192423 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:00:07.192423 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:00:07.192423 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:00:07.192423 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:00:07.192423 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:00:07.192423 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:00:07.192423 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:00:07.192423 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:00:07.192423 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:00:07.192423 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:00:07.192423 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 20:00:07.192423 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 20:00:07.192423 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 20:00:07.195125 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 20:00:07.195125 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:00:07.195125 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:00:07.195125 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:00:07.195125 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:00:07.195125 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:00:07.195125 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:00:07.195125 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:00:07.195125 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 20:00:07.195125 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:00:07.195125 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:00:07.195125 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:00:07.195125 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:00:07.195125 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:00:07.195125 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:00:07.195125 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:00:07.195125 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:00:07.196727 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 20:00:07.196727 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 20:00:07.196727 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 20:00:07.196727 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 20:00:07.196727 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 20:00:07.196727 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 20:00:07.196727 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 20:00:07.196727 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 20:00:07.196727 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 417/802] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 6 20:00:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 367 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:00:07.203400 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7462d0bb-5175-49ae-b1b7-68ad59f20041 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:07.246063 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:07.280187 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26c6c67-ee71-4800-946b-542d4aca7be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:07.281337 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26c6c67-ee71-4800-946b-542d4aca7be5 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:07.281817 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26c6c67-ee71-4800-946b-542d4aca7be5 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:07.282297 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26c6c67-ee71-4800-946b-542d4aca7be5 None admin] PATH_INFO: `/v3/domains` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:07.287589 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26c6c67-ee71-4800-946b-542d4aca7be5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:07.288135 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26c6c67-ee71-4800-946b-542d4aca7be5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:07.290816 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26c6c67-ee71-4800-946b-542d4aca7be5 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:07.330486 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 386/803] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 6 20:00:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 432 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:00:07.357063 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26c6c67-ee71-4800-946b-542d4aca7be5 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:07.371104 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:07.492432 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26c6c67-ee71-4800-946b-542d4aca7be5 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:07.510503 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7462d0bb-5175-49ae-b1b7-68ad59f20041 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:07.543650 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae79614-109a-4180-96ab-d696f2bb7346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:07.544431 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae79614-109a-4180-96ab-d696f2bb7346 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:07.544725 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae79614-109a-4180-96ab-d696f2bb7346 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:07.544991 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae79614-109a-4180-96ab-d696f2bb7346 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:07.554323 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae79614-109a-4180-96ab-d696f2bb7346 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:07.555858 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae79614-109a-4180-96ab-d696f2bb7346 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:07.577231 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-7ae79614-109a-4180-96ab-d696f2bb7346 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 20:00:07.577231 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:07.577231 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:00:07.577231 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:00:07.577231 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:00:07.577231 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:00:07.577231 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:00:07.577231 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:00:07.577231 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:00:07.577231 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:00:07.577231 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:00:07.577231 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:00:07.577231 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 20:00:07.577231 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 20:00:07.577231 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 20:00:07.579008 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 20:00:07.579008 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:00:07.579008 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:00:07.579008 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:00:07.579008 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:00:07.579008 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:00:07.579008 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:00:07.579008 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:00:07.579008 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 20:00:07.579008 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:00:07.579008 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:00:07.579008 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:00:07.579008 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:00:07.579008 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:00:07.579008 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:00:07.579008 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:00:07.579008 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:00:07.580798 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 20:00:07.580798 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 20:00:07.580798 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 20:00:07.580798 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 20:00:07.580798 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 20:00:07.580798 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 20:00:07.580798 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 20:00:07.580798 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:00:07.580798 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 387/804] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 6 20:00:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 20:00:07.580798 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae79614-109a-4180-96ab-d696f2bb7346 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:07.612301 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f95485d-d5c5-4f80-9ae7-73540db5d76a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:07.613648 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f95485d-d5c5-4f80-9ae7-73540db5d76a None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:07.614129 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f95485d-d5c5-4f80-9ae7-73540db5d76a None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:07.614616 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f95485d-d5c5-4f80-9ae7-73540db5d76a None admin] PATH_INFO: `/v3/projects` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:07.616959 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f95485d-d5c5-4f80-9ae7-73540db5d76a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:07.616959 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f95485d-d5c5-4f80-9ae7-73540db5d76a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:07.619829 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f95485d-d5c5-4f80-9ae7-73540db5d76a None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:07.657892 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 418/805] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Sep 6 20:00:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 450 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:00:07.659801 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f95485d-d5c5-4f80-9ae7-73540db5d76a None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:07.731437 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:07.816125 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:07.823034 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae79614-109a-4180-96ab-d696f2bb7346 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:07.889292 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f95485d-d5c5-4f80-9ae7-73540db5d76a None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:07.898335 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac74a244-2188-4e3f-b3de-0d0d82c7412f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:07.899953 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac74a244-2188-4e3f-b3de-0d0d82c7412f None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:07.900751 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac74a244-2188-4e3f-b3de-0d0d82c7412f None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:07.901542 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac74a244-2188-4e3f-b3de-0d0d82c7412f None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:07.903037 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac74a244-2188-4e3f-b3de-0d0d82c7412f None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:07.906400 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac74a244-2188-4e3f-b3de-0d0d82c7412f None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:07.927512 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-ac74a244-2188-4e3f-b3de-0d0d82c7412f None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 06 20:00:07.927512 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:07.927512 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 20:00:07.927512 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 20:00:07.927512 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 20:00:07.927512 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 20:00:07.927512 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 20:00:07.927512 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 20:00:07.927512 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 06 20:00:07.927512 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:00:07.927512 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 20:00:07.927512 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:00:07.927512 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:07.927512 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:00:07.927512 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:00:07.927512 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:00:07.927512 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:00:07.929449 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:00:07.929449 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:00:07.929449 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:00:07.929449 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:00:07.929449 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:00:07.929449 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:00:07.929449 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 20:00:07.929449 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 20:00:07.929449 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 20:00:07.929449 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 20:00:07.929449 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:00:07.929449 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:00:07.929449 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:00:07.929449 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:00:07.929449 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:00:07.929449 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:00:07.931170 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:00:07.931170 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 20:00:07.931170 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:00:07.931170 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:00:07.931170 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:00:07.931170 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:00:07.931170 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:00:07.931170 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:00:07.931170 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:00:07.931170 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:00:07.931170 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 20:00:07.931170 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 20:00:07.931170 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 06 20:00:07.931170 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:00:07.942098 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 388/806] 10.222.0.20 () {56 vars in 1129 bytes} [Tue Sep 6 20:00:07 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 357 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:00:07.944078 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac74a244-2188-4e3f-b3de-0d0d82c7412f None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:07.963693 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdb21be-7882-4a32-804c-4b9b5ecb5f54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:07.964761 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdb21be-7882-4a32-804c-4b9b5ecb5f54 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:07.965247 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdb21be-7882-4a32-804c-4b9b5ecb5f54 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:07.965782 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdb21be-7882-4a32-804c-4b9b5ecb5f54 None admin] PATH_INFO: `/v3/projects` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:07.967058 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdb21be-7882-4a32-804c-4b9b5ecb5f54 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:07.967555 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdb21be-7882-4a32-804c-4b9b5ecb5f54 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:07.970422 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdb21be-7882-4a32-804c-4b9b5ecb5f54 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:07.999856 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 419/807] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Sep 6 20:00:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:00:08.000645 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdb21be-7882-4a32-804c-4b9b5ecb5f54 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:08.175383 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac74a244-2188-4e3f-b3de-0d0d82c7412f None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:08.212973 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdb21be-7882-4a32-804c-4b9b5ecb5f54 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:08.231925 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:08.249464 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4863d6-5822-462d-bf63-5ede6cdcb1ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:08.251361 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4863d6-5822-462d-bf63-5ede6cdcb1ef None admin] REQUEST_METHOD: `PUT` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:08.251361 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4863d6-5822-462d-bf63-5ede6cdcb1ef None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:08.251361 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4863d6-5822-462d-bf63-5ede6cdcb1ef None admin] PATH_INFO: `/v3/projects/6aef43493bb1420a9e122313d8db7d5a/users/1a52117df70447dba9bbee35d083e44f/roles/b950bf73c93e4f7ebed923ff14e39faa` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:08.267106 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:08.275323 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4863d6-5822-462d-bf63-5ede6cdcb1ef None admin] RBAC: Authorizing `identity:create_grant(project_id=6aef43493bb1420a9e122313d8db7d5a, user_id=1a52117df70447dba9bbee35d083e44f, role_id=b950bf73c93e4f7ebed923ff14e39faa)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:08.276660 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6c5632-9c72-46c8-9cbb-4c0c398ce4a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:08.277589 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4863d6-5822-462d-bf63-5ede6cdcb1ef None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:08.277803 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6c5632-9c72-46c8-9cbb-4c0c398ce4a9 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:08.278282 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6c5632-9c72-46c8-9cbb-4c0c398ce4a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:08.278761 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6c5632-9c72-46c8-9cbb-4c0c398ce4a9 None admin] PATH_INFO: `/v3/services` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:08.279923 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6c5632-9c72-46c8-9cbb-4c0c398ce4a9 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:08.280925 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6c5632-9c72-46c8-9cbb-4c0c398ce4a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:08.285559 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6c5632-9c72-46c8-9cbb-4c0c398ce4a9 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:08.294266 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 420/808] 10.222.0.20 () {56 vars in 1124 bytes} [Tue Sep 6 20:00:07 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:00:08.295483 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6c5632-9c72-46c8-9cbb-4c0c398ce4a9 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:08.311003 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 389/809] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Sep 6 20:00:07 2022] PUT /identity/v3/projects/6aef43493bb1420a9e122313d8db7d5a/users/1a52117df70447dba9bbee35d083e44f/roles/b950bf73c93e4f7ebed923ff14e39faa => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:00:08.311003 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be74070-c1a1-466f-8f2f-b89be4089798 None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:08.311003 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be74070-c1a1-466f-8f2f-b89be4089798 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:08.311003 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be74070-c1a1-466f-8f2f-b89be4089798 None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:08.311003 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 390/810] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:00:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:00:08.311003 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be74070-c1a1-466f-8f2f-b89be4089798 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:08.505792 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be74070-c1a1-466f-8f2f-b89be4089798 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:08.539639 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6c5632-9c72-46c8-9cbb-4c0c398ce4a9 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:08.588325 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4172fc2e-89ba-4955-94b3-1025edd5faed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:08.589546 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4172fc2e-89ba-4955-94b3-1025edd5faed None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:08.590045 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4172fc2e-89ba-4955-94b3-1025edd5faed None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:08.590482 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4172fc2e-89ba-4955-94b3-1025edd5faed None admin] PATH_INFO: `/v3/services` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:08.591480 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4172fc2e-89ba-4955-94b3-1025edd5faed None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:08.591947 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4172fc2e-89ba-4955-94b3-1025edd5faed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:08.593632 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4172fc2e-89ba-4955-94b3-1025edd5faed None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:08.600404 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 391/811] 10.222.0.20 () {56 vars in 1124 bytes} [Tue Sep 6 20:00:08 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:00:08.602165 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392756bf-f26e-4fbc-8751-82db2a1466a6 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:08.602727 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392756bf-f26e-4fbc-8751-82db2a1466a6 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:08.603133 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392756bf-f26e-4fbc-8751-82db2a1466a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:08.606388 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b53eb7a-1dce-4995-9b0d-0fb32c0264b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:08.607777 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b53eb7a-1dce-4995-9b0d-0fb32c0264b3 None admin] REQUEST_METHOD: `PUT` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:08.608828 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b53eb7a-1dce-4995-9b0d-0fb32c0264b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:08.609468 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b53eb7a-1dce-4995-9b0d-0fb32c0264b3 None admin] PATH_INFO: `/v3/projects/6aef43493bb1420a9e122313d8db7d5a/users/2b72516236da47e292d78de201b0552b/roles/e0ab41d8ae22474688b5361bcc75e0f0` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:08.615580 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b53eb7a-1dce-4995-9b0d-0fb32c0264b3 None admin] RBAC: Authorizing `identity:create_grant(project_id=6aef43493bb1420a9e122313d8db7d5a, user_id=2b72516236da47e292d78de201b0552b, role_id=e0ab41d8ae22474688b5361bcc75e0f0)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:08.618649 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b53eb7a-1dce-4995-9b0d-0fb32c0264b3 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:08.629250 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-392756bf-f26e-4fbc-8751-82db2a1466a6 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:00:08.631244 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-392756bf-f26e-4fbc-8751-82db2a1466a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:00:08.652867 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 421/812] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Sep 6 20:00:08 2022] PUT /identity/v3/projects/6aef43493bb1420a9e122313d8db7d5a/users/2b72516236da47e292d78de201b0552b/roles/e0ab41d8ae22474688b5361bcc75e0f0 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:00:08.658003 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b53eb7a-1dce-4995-9b0d-0fb32c0264b3 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:08.721660 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:08.810680 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 392/813] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:00:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 20:00:08.817249 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:08.828378 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25f8042-a133-4c1c-bc26-8249caf75b6a None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:08.828676 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25f8042-a133-4c1c-bc26-8249caf75b6a None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:08.828922 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25f8042-a133-4c1c-bc26-8249caf75b6a None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:08.830858 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 393/814] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:00:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 20:00:08.836899 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380e368f-8ae8-447d-92e8-7fad79771123 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:08.837187 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380e368f-8ae8-447d-92e8-7fad79771123 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:08.837444 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380e368f-8ae8-447d-92e8-7fad79771123 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:08.859500 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-380e368f-8ae8-447d-92e8-7fad79771123 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:00:08.861010 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-380e368f-8ae8-447d-92e8-7fad79771123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:00:08.908901 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b53eb7a-1dce-4995-9b0d-0fb32c0264b3 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:08.991229 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80dff10-6917-46b9-b875-f183c8d5540c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:08.992087 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80dff10-6917-46b9-b875-f183c8d5540c None admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:08.992373 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80dff10-6917-46b9-b875-f183c8d5540c None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:08.992652 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80dff10-6917-46b9-b875-f183c8d5540c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:08.993366 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80dff10-6917-46b9-b875-f183c8d5540c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:08.995043 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80dff10-6917-46b9-b875-f183c8d5540c None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:09.043980 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 422/815] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:00:08 2022] POST /identity/v3/endpoints => generated 346 bytes in 390 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:00:09.076997 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 394/816] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:00:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 20:00:09.274433 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80dff10-6917-46b9-b875-f183c8d5540c None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:09.480131 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80dff10-6917-46b9-b875-f183c8d5540c None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:09.503696 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:09.510836 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380e368f-8ae8-447d-92e8-7fad79771123 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:09.554342 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae5d7d6-e49b-4af2-9fd6-399e1ce9f947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:09.555104 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae5d7d6-e49b-4af2-9fd6-399e1ce9f947 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:09.555608 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae5d7d6-e49b-4af2-9fd6-399e1ce9f947 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:09.556062 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae5d7d6-e49b-4af2-9fd6-399e1ce9f947 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:09.584536 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae5d7d6-e49b-4af2-9fd6-399e1ce9f947 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 20:00:09.595890 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-1ae5d7d6-e49b-4af2-9fd6-399e1ce9f947 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 20:00:09.595890 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:09.595890 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:00:09.595890 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:00:09.595890 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:00:09.595890 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:00:09.595890 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:00:09.595890 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:00:09.595890 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:00:09.595890 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:00:09.595890 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:00:09.595890 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:00:09.595890 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 20:00:09.595890 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 20:00:09.595890 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 20:00:09.595890 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 20:00:09.598238 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 20:00:09.598238 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 20:00:09.598238 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 20:00:09.598238 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 20:00:09.598238 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:00:09.598238 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:00:09.598238 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:00:09.598238 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:00:09.598238 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:00:09.598238 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:00:09.598238 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:00:09.598238 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 20:00:09.598238 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:00:09.598238 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:00:09.598238 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:00:09.598238 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:00:09.600298 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:00:09.600298 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:00:09.600298 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:00:09.600298 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:00:09.600298 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 20:00:09.600298 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 20:00:09.600298 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 20:00:09.600298 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 20:00:09.600298 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 20:00:09.600298 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 20:00:09.600298 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 20:00:09.600298 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 20:00:09.604012 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 423/817] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Sep 6 20:00:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:00:09.613488 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae5d7d6-e49b-4af2-9fd6-399e1ce9f947 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:09.755337 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:09.767193 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380e368f-8ae8-447d-92e8-7fad79771123 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:09.843382 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04f2f5b-33bb-4e48-ae7d-a2b9d06abe9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:09.844120 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04f2f5b-33bb-4e48-ae7d-a2b9d06abe9b None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:09.844406 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04f2f5b-33bb-4e48-ae7d-a2b9d06abe9b None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:09.844716 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04f2f5b-33bb-4e48-ae7d-a2b9d06abe9b None admin] PATH_INFO: `/v3/services/network` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:09.845346 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04f2f5b-33bb-4e48-ae7d-a2b9d06abe9b None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:09.847055 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04f2f5b-33bb-4e48-ae7d-a2b9d06abe9b None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:09.854025 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae5d7d6-e49b-4af2-9fd6-399e1ce9f947 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:09.871150 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-d04f2f5b-33bb-4e48-ae7d-a2b9d06abe9b None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 06 20:00:09.871150 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:09.871150 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 20:00:09.871150 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 20:00:09.871150 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 20:00:09.871150 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 20:00:09.871150 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 20:00:09.871150 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 20:00:09.871150 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 06 20:00:09.871150 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:00:09.871150 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 20:00:09.871150 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:00:09.871150 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:09.871150 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:00:09.871150 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:00:09.871150 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:00:09.871150 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:00:09.873106 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:00:09.873106 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:00:09.873106 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:00:09.873106 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:00:09.873106 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:00:09.873106 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:00:09.873106 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 20:00:09.873106 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 20:00:09.873106 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 20:00:09.873106 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 20:00:09.873106 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:00:09.873106 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:00:09.873106 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:00:09.873106 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:00:09.873106 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:00:09.873106 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:00:09.875355 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:00:09.875355 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 20:00:09.875355 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:00:09.875355 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:00:09.875355 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:00:09.875355 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:00:09.875355 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:00:09.875355 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:00:09.875355 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:00:09.875355 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:00:09.875355 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 20:00:09.875355 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 20:00:09.875355 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 06 20:00:09.875355 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:00:09.875355 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 395/818] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 6 20:00:09 2022] GET /identity/v3/services/network => generated 88 bytes in 363 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:00:09.898784 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04f2f5b-33bb-4e48-ae7d-a2b9d06abe9b None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:09.968327 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82befb6-e465-414e-90c6-69c76ed4e8f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:09.970100 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82befb6-e465-414e-90c6-69c76ed4e8f0 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:09.971889 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82befb6-e465-414e-90c6-69c76ed4e8f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:09.972475 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82befb6-e465-414e-90c6-69c76ed4e8f0 None admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:09.973800 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82befb6-e465-414e-90c6-69c76ed4e8f0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:09.974473 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82befb6-e465-414e-90c6-69c76ed4e8f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:09.983883 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82befb6-e465-414e-90c6-69c76ed4e8f0 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:10.004446 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 424/819] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 6 20:00:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 392 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 20:00:10.015716 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82befb6-e465-414e-90c6-69c76ed4e8f0 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:10.114251 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:10.204237 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04f2f5b-33bb-4e48-ae7d-a2b9d06abe9b None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:10.250279 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466b2648-09f8-461f-bd33-a3ec995a0a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:10.253366 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466b2648-09f8-461f-bd33-a3ec995a0a47 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:10.254157 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466b2648-09f8-461f-bd33-a3ec995a0a47 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:10.254450 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466b2648-09f8-461f-bd33-a3ec995a0a47 None admin] PATH_INFO: `/v3/services` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:10.255273 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466b2648-09f8-461f-bd33-a3ec995a0a47 None admin] RBAC: Adding query filter params (name=network) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:10.255605 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466b2648-09f8-461f-bd33-a3ec995a0a47 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:10.257207 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466b2648-09f8-461f-bd33-a3ec995a0a47 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:10.272274 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 396/820] 10.222.0.20 () {56 vars in 1112 bytes} [Tue Sep 6 20:00:09 2022] GET /identity/v3/services?name=network => generated 124 bytes in 374 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:00:10.275764 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82befb6-e465-414e-90c6-69c76ed4e8f0 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:10.287428 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466b2648-09f8-461f-bd33-a3ec995a0a47 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:10.362071 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:10.367813 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d969e6-3e0f-4a3b-bc18-fb1627a01ae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:10.368863 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d969e6-3e0f-4a3b-bc18-fb1627a01ae4 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:10.369346 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d969e6-3e0f-4a3b-bc18-fb1627a01ae4 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:10.369860 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d969e6-3e0f-4a3b-bc18-fb1627a01ae4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:10.412810 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d969e6-3e0f-4a3b-bc18-fb1627a01ae4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:10.415161 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d969e6-3e0f-4a3b-bc18-fb1627a01ae4 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:10.443217 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466b2648-09f8-461f-bd33-a3ec995a0a47 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:10.483247 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-47d969e6-3e0f-4a3b-bc18-fb1627a01ae4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 20:00:10.483247 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:10.483247 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:00:10.483247 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:00:10.483247 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:00:10.483247 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:00:10.483247 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:00:10.483247 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:00:10.483247 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:00:10.483247 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:00:10.483247 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:00:10.483247 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:00:10.483247 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 20:00:10.483247 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 20:00:10.483247 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 20:00:10.483247 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 20:00:10.485102 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:00:10.485102 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:00:10.485102 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:00:10.485102 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:00:10.485102 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:00:10.485102 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:00:10.485102 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:00:10.485102 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 20:00:10.485102 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:00:10.485102 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:00:10.485102 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:00:10.485102 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:00:10.485102 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:00:10.485102 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:00:10.485102 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:00:10.485102 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:00:10.485102 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 20:00:10.486885 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 20:00:10.486885 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 20:00:10.486885 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 20:00:10.486885 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 425/821] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 20:00:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 472 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:00:10.487298 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e80d70-13f8-402e-b19e-6493111052d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:10.488203 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e80d70-13f8-402e-b19e-6493111052d0 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:10.488203 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e80d70-13f8-402e-b19e-6493111052d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:10.488203 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e80d70-13f8-402e-b19e-6493111052d0 None admin] PATH_INFO: `/v3/services` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:10.488856 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e80d70-13f8-402e-b19e-6493111052d0 None admin] RBAC: Adding query filter params (type=network) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:10.489082 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e80d70-13f8-402e-b19e-6493111052d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:10.496414 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e80d70-13f8-402e-b19e-6493111052d0 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:10.497809 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 397/822] 10.222.0.20 () {56 vars in 1112 bytes} [Tue Sep 6 20:00:10 2022] GET /identity/v3/services?type=network => generated 350 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:00:10.499733 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d969e6-3e0f-4a3b-bc18-fb1627a01ae4 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:10.517670 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e80d70-13f8-402e-b19e-6493111052d0 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:10.743408 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:10.778985 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:10.806882 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e80d70-13f8-402e-b19e-6493111052d0 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:10.818091 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d969e6-3e0f-4a3b-bc18-fb1627a01ae4 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:10.884465 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa552cb2-92e1-4776-9549-07d088a7877d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:10.885548 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa552cb2-92e1-4776-9549-07d088a7877d None admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:10.886227 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa552cb2-92e1-4776-9549-07d088a7877d None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:10.886718 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa552cb2-92e1-4776-9549-07d088a7877d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:10.887999 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa552cb2-92e1-4776-9549-07d088a7877d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:10.891386 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa552cb2-92e1-4776-9549-07d088a7877d None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:10.937909 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd46e8a-4a06-4fe6-9014-cadab96b0cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:10.938889 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd46e8a-4a06-4fe6-9014-cadab96b0cf0 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:10.939302 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd46e8a-4a06-4fe6-9014-cadab96b0cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:10.939706 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd46e8a-4a06-4fe6-9014-cadab96b0cf0 None admin] PATH_INFO: `/v3/domains` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:10.940745 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd46e8a-4a06-4fe6-9014-cadab96b0cf0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:10.941239 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd46e8a-4a06-4fe6-9014-cadab96b0cf0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:10.943906 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd46e8a-4a06-4fe6-9014-cadab96b0cf0 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:10.968719 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 398/823] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:00:10 2022] POST /identity/v3/endpoints => generated 337 bytes in 452 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:00:10.986063 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 426/824] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 6 20:00:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 488 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:00:10.992330 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa552cb2-92e1-4776-9549-07d088a7877d None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:11.208868 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa552cb2-92e1-4776-9549-07d088a7877d None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:11.266696 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a1e81b-3678-40cc-8634-d8ef9bf73383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:11.268301 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a1e81b-3678-40cc-8634-d8ef9bf73383 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:11.269088 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a1e81b-3678-40cc-8634-d8ef9bf73383 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:11.269888 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a1e81b-3678-40cc-8634-d8ef9bf73383 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:11.286530 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a1e81b-3678-40cc-8634-d8ef9bf73383 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:11.289953 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a1e81b-3678-40cc-8634-d8ef9bf73383 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:11.315021 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-99a1e81b-3678-40cc-8634-d8ef9bf73383 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 06 20:00:11.315021 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:11.315021 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:00:11.315021 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:00:11.315021 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:00:11.315021 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:00:11.315021 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:00:11.315021 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:00:11.315021 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:00:11.315021 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:00:11.315021 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:00:11.315021 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:00:11.315021 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 20:00:11.315021 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 20:00:11.315021 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 20:00:11.315021 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 20:00:11.316927 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:00:11.316927 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:00:11.316927 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 20:00:11.316927 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 20:00:11.316927 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 20:00:11.316927 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 20:00:11.316927 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:00:11.316927 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:00:11.316927 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:00:11.316927 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:00:11.316927 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:00:11.316927 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 20:00:11.316927 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:00:11.316927 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:00:11.316927 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:00:11.316927 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:00:11.316927 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:00:11.316927 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:00:11.318930 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:00:11.318930 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:00:11.318930 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 20:00:11.318930 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 20:00:11.318930 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 20:00:11.318930 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 20:00:11.318930 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 20:00:11.318930 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 20:00:11.318930 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 06 20:00:11.318930 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:00:11.323798 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 399/825] 10.222.0.20 () {56 vars in 1093 bytes} [Tue Sep 6 20:00:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:00:11.332996 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd46e8a-4a06-4fe6-9014-cadab96b0cf0 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:11.564733 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd16a035-0317-4c03-868c-50d359fd49f5 None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:11.566063 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd16a035-0317-4c03-868c-50d359fd49f5 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:11.566826 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd16a035-0317-4c03-868c-50d359fd49f5 None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:11.577923 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 400/826] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:00:11 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:00:11.583704 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfa9cf4-68c5-4610-ba0c-75f04b7f6a84 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:11.584133 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfa9cf4-68c5-4610-ba0c-75f04b7f6a84 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:11.584519 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfa9cf4-68c5-4610-ba0c-75f04b7f6a84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:11.609377 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:11.619011 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd46e8a-4a06-4fe6-9014-cadab96b0cf0 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:11.627372 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-7cfa9cf4-68c5-4610-ba0c-75f04b7f6a84 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:00:11.629519 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-7cfa9cf4-68c5-4610-ba0c-75f04b7f6a84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:00:11.681225 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a188910-6970-40d4-a717-1f88ad0c25d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:11.681982 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a188910-6970-40d4-a717-1f88ad0c25d7 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:11.682265 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a188910-6970-40d4-a717-1f88ad0c25d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:11.682536 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a188910-6970-40d4-a717-1f88ad0c25d7 None admin] PATH_INFO: `/v3/users` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:11.683421 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a188910-6970-40d4-a717-1f88ad0c25d7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:11.683779 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a188910-6970-40d4-a717-1f88ad0c25d7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:11.685363 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a188910-6970-40d4-a717-1f88ad0c25d7 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:11.727846 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 427/827] 10.222.0.20 () {56 vars in 1133 bytes} [Tue Sep 6 20:00:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 396 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:00:11.735468 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a188910-6970-40d4-a717-1f88ad0c25d7 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:11.742589 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:11.785766 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 401/828] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:00:11 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 20:00:11.954454 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a188910-6970-40d4-a717-1f88ad0c25d7 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:12.024991 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9417d76-fcc0-4232-8ae2-3d535e3bc888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:12.026107 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9417d76-fcc0-4232-8ae2-3d535e3bc888 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:12.026610 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9417d76-fcc0-4232-8ae2-3d535e3bc888 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:12.027096 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9417d76-fcc0-4232-8ae2-3d535e3bc888 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:12.160428 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9417d76-fcc0-4232-8ae2-3d535e3bc888 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:12.163305 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9417d76-fcc0-4232-8ae2-3d535e3bc888 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:12.168782 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:12.324839 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfa9cf4-68c5-4610-ba0c-75f04b7f6a84 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:12.382158 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 428/829] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 6 20:00:11 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 648 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 06 20:00:12.386511 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9417d76-fcc0-4232-8ae2-3d535e3bc888 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:12.562032 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfa9cf4-68c5-4610-ba0c-75f04b7f6a84 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:12.611420 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9417d76-fcc0-4232-8ae2-3d535e3bc888 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:12.616119 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:12.630981 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:12.643597 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d53b71-d43e-484c-b218-7c85318cbe72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:12.644797 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d53b71-d43e-484c-b218-7c85318cbe72 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:12.645271 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d53b71-d43e-484c-b218-7c85318cbe72 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:12.645795 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d53b71-d43e-484c-b218-7c85318cbe72 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:12.667660 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d53b71-d43e-484c-b218-7c85318cbe72 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 20:00:12.677661 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-91d53b71-d43e-484c-b218-7c85318cbe72 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 20:00:12.677661 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:12.677661 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:00:12.677661 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:00:12.677661 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:00:12.677661 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:00:12.677661 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:00:12.677661 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:00:12.677661 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:00:12.677661 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:00:12.677661 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:00:12.677661 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:00:12.677661 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 20:00:12.677661 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 20:00:12.677661 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 20:00:12.677661 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 20:00:12.679048 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 20:00:12.679048 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 20:00:12.679048 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 20:00:12.679048 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 20:00:12.679048 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:00:12.679048 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:00:12.679048 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:00:12.679048 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:00:12.679048 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:00:12.679048 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:00:12.679048 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:00:12.679048 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 20:00:12.679048 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:00:12.679048 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:00:12.679048 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:00:12.679048 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:00:12.680123 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:00:12.680123 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:00:12.680123 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:00:12.680123 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:00:12.680123 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 20:00:12.680123 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 20:00:12.680123 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 20:00:12.680123 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 20:00:12.680123 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 20:00:12.680123 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 20:00:12.680123 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 20:00:12.680123 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:00:12.685500 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a798c6-e656-4591-b7f3-417c3bf6756e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:12.687062 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a798c6-e656-4591-b7f3-417c3bf6756e None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:12.687482 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a798c6-e656-4591-b7f3-417c3bf6756e None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:12.687899 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a798c6-e656-4591-b7f3-417c3bf6756e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:12.689062 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 402/830] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Sep 6 20:00:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 365 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 20:00:12.691370 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9f4f93-2344-481a-8307-696179449f5c None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:12.691863 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9f4f93-2344-481a-8307-696179449f5c None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:12.692358 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9f4f93-2344-481a-8307-696179449f5c None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:12.695045 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 403/831] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:00:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:00:12.696562 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9f4f93-2344-481a-8307-696179449f5c None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:12.709810 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a798c6-e656-4591-b7f3-417c3bf6756e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:12.713050 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a798c6-e656-4591-b7f3-417c3bf6756e None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:12.907297 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9f4f93-2344-481a-8307-696179449f5c None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:12.923198 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-e5a798c6-e656-4591-b7f3-417c3bf6756e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 20:00:12.923198 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:12.923198 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:00:12.923198 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:00:12.923198 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:00:12.923198 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:00:12.923198 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:00:12.923198 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:00:12.923198 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:00:12.923198 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:00:12.923198 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:00:12.923198 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:00:12.923198 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 20:00:12.923198 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 20:00:12.923198 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 20:00:12.923198 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 20:00:12.925332 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:00:12.925332 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:00:12.925332 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:00:12.925332 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:00:12.925332 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:00:12.925332 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:00:12.925332 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:00:12.925332 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 20:00:12.925332 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:00:12.925332 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:00:12.925332 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:00:12.925332 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:00:12.925332 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:00:12.925332 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:00:12.925332 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:00:12.925332 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:00:12.925332 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 20:00:12.926575 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 20:00:12.926575 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 20:00:12.926575 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 20:00:12.926575 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 429/832] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 20:00:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 540 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:00:12.927838 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a9c652-07ea-4391-9fef-e2b6bbb2d045 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:12.928341 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a9c652-07ea-4391-9fef-e2b6bbb2d045 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:12.928957 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a9c652-07ea-4391-9fef-e2b6bbb2d045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:12.952573 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-91a9c652-07ea-4391-9fef-e2b6bbb2d045 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:00:12.953965 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-91a9c652-07ea-4391-9fef-e2b6bbb2d045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:00:12.983327 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae45082-5482-46f1-9d5a-931ef42a8787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:12.984388 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae45082-5482-46f1-9d5a-931ef42a8787 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:12.984923 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae45082-5482-46f1-9d5a-931ef42a8787 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:12.985410 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae45082-5482-46f1-9d5a-931ef42a8787 None admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:12.986768 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae45082-5482-46f1-9d5a-931ef42a8787 None admin] RBAC: Adding query filter params (name=service) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:12.987400 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae45082-5482-46f1-9d5a-931ef42a8787 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:12.990244 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae45082-5482-46f1-9d5a-931ef42a8787 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:13.007160 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 404/833] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Sep 6 20:00:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:00:13.009573 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc7d9b0-33cc-421a-9e02-ce16d419b712 None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:13.010105 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc7d9b0-33cc-421a-9e02-ce16d419b712 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:13.010678 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc7d9b0-33cc-421a-9e02-ce16d419b712 None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:13.014392 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 405/834] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:00:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:00:13.016113 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc7d9b0-33cc-421a-9e02-ce16d419b712 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:13.037637 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 430/835] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:00:12 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 20:00:13.038608 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a9c652-07ea-4391-9fef-e2b6bbb2d045 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:13.135934 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:13.144856 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:13.171158 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a9c652-07ea-4391-9fef-e2b6bbb2d045 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:13.213933 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc7d9b0-33cc-421a-9e02-ce16d419b712 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:13.221071 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a072ce8-56db-41c7-8afb-b7cd69ebbe6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:13.222112 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a072ce8-56db-41c7-8afb-b7cd69ebbe6a None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:13.222548 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a072ce8-56db-41c7-8afb-b7cd69ebbe6a None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:13.222989 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a072ce8-56db-41c7-8afb-b7cd69ebbe6a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:13.241081 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a072ce8-56db-41c7-8afb-b7cd69ebbe6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:13.242797 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a072ce8-56db-41c7-8afb-b7cd69ebbe6a None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:13.261325 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-0a072ce8-56db-41c7-8afb-b7cd69ebbe6a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 20:00:13.261325 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:13.261325 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:00:13.261325 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:00:13.261325 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:00:13.261325 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:00:13.261325 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:00:13.261325 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:00:13.261325 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:00:13.261325 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:00:13.261325 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:00:13.261325 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:00:13.261325 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 20:00:13.261325 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 20:00:13.261325 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 20:00:13.261325 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 20:00:13.263202 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:00:13.263202 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:00:13.263202 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:00:13.263202 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:00:13.263202 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:00:13.263202 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:00:13.263202 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:00:13.263202 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 20:00:13.263202 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:00:13.263202 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:00:13.263202 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:00:13.263202 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:00:13.263202 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:00:13.263202 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:00:13.263202 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:00:13.263202 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:00:13.263202 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 20:00:13.266815 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 20:00:13.266815 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 20:00:13.266815 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 20:00:13.266815 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 431/836] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 20:00:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:00:13.266815 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797d1fb4-9950-46d6-9396-061143bb3df5 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:13.266815 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797d1fb4-9950-46d6-9396-061143bb3df5 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:13.266815 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797d1fb4-9950-46d6-9396-061143bb3df5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:13.276176 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a1de1a-31bc-4bac-9996-c84c1c81f905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:13.276986 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a1de1a-31bc-4bac-9996-c84c1c81f905 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:13.277389 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a1de1a-31bc-4bac-9996-c84c1c81f905 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:13.277850 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a1de1a-31bc-4bac-9996-c84c1c81f905 None admin] PATH_INFO: `/v3/domains` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:13.278665 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a1de1a-31bc-4bac-9996-c84c1c81f905 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:13.279106 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a1de1a-31bc-4bac-9996-c84c1c81f905 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:13.281171 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a1de1a-31bc-4bac-9996-c84c1c81f905 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:13.287386 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-797d1fb4-9950-46d6-9396-061143bb3df5 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:00:13.288609 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-797d1fb4-9950-46d6-9396-061143bb3df5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:00:13.297540 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 406/837] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 6 20:00:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:00:13.302709 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a1de1a-31bc-4bac-9996-c84c1c81f905 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:13.376065 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 432/838] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:00:13 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 20:00:13.377481 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797d1fb4-9950-46d6-9396-061143bb3df5 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:13.462854 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a1de1a-31bc-4bac-9996-c84c1c81f905 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:13.482937 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:13.494717 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:13.530329 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fe3179-30d6-4d63-8e30-7c5799ff0d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:13.531062 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fe3179-30d6-4d63-8e30-7c5799ff0d00 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:13.531369 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fe3179-30d6-4d63-8e30-7c5799ff0d00 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:13.531691 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fe3179-30d6-4d63-8e30-7c5799ff0d00 None admin] PATH_INFO: `/v3/domains` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:13.532567 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fe3179-30d6-4d63-8e30-7c5799ff0d00 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:13.532972 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fe3179-30d6-4d63-8e30-7c5799ff0d00 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:13.534648 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fe3179-30d6-4d63-8e30-7c5799ff0d00 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:13.549966 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797d1fb4-9950-46d6-9396-061143bb3df5 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:13.552073 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 407/839] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 6 20:00:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:00:13.552827 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fe3179-30d6-4d63-8e30-7c5799ff0d00 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:13.629011 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d3d6e4-ffeb-4a80-883f-2250def0d962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:13.630156 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d3d6e4-ffeb-4a80-883f-2250def0d962 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:13.630839 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d3d6e4-ffeb-4a80-883f-2250def0d962 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:13.631307 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d3d6e4-ffeb-4a80-883f-2250def0d962 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:13.644382 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d3d6e4-ffeb-4a80-883f-2250def0d962 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:13.647196 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d3d6e4-ffeb-4a80-883f-2250def0d962 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:13.670181 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-03d3d6e4-ffeb-4a80-883f-2250def0d962 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 20:00:13.670181 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:13.670181 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:00:13.670181 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:00:13.670181 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:00:13.670181 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:00:13.670181 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:00:13.670181 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:00:13.670181 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:00:13.670181 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:00:13.670181 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:00:13.670181 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:00:13.670181 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 20:00:13.670181 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 20:00:13.670181 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 20:00:13.671389 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 20:00:13.671389 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:00:13.671389 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:00:13.671389 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:00:13.671389 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:00:13.671389 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:00:13.671389 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:00:13.671389 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:00:13.671389 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 20:00:13.671389 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:00:13.671389 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:00:13.671389 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:00:13.671389 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:00:13.671389 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:00:13.671389 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:00:13.671389 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:00:13.671389 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:00:13.672582 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 20:00:13.672582 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 20:00:13.672582 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 20:00:13.672582 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 20:00:13.672582 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 20:00:13.672582 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 20:00:13.672582 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 20:00:13.672582 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 20:00:13.678520 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 433/840] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 6 20:00:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:00:13.680280 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d3d6e4-ffeb-4a80-883f-2250def0d962 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:13.709148 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fe3179-30d6-4d63-8e30-7c5799ff0d00 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:13.758606 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ec7426-6b7a-4c52-ab22-bd9260770cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:13.759307 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ec7426-6b7a-4c52-ab22-bd9260770cfd None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:13.759674 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ec7426-6b7a-4c52-ab22-bd9260770cfd None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:13.760032 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ec7426-6b7a-4c52-ab22-bd9260770cfd None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:13.760750 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ec7426-6b7a-4c52-ab22-bd9260770cfd None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:13.762454 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ec7426-6b7a-4c52-ab22-bd9260770cfd None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:13.776574 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-71ec7426-6b7a-4c52-ab22-bd9260770cfd None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 06 20:00:13.776574 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:13.776574 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 20:00:13.776574 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 20:00:13.776574 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 20:00:13.776574 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 20:00:13.776574 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 20:00:13.776574 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 20:00:13.776574 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 06 20:00:13.776574 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:00:13.776574 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 20:00:13.776574 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:00:13.776574 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:13.776574 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:00:13.776574 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:00:13.776574 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:00:13.776574 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:00:13.778607 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:00:13.778607 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:00:13.778607 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:00:13.778607 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:00:13.778607 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:00:13.778607 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:00:13.778607 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 20:00:13.778607 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 20:00:13.778607 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 20:00:13.778607 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 20:00:13.778607 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:00:13.778607 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:00:13.778607 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:00:13.778607 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:00:13.778607 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:00:13.778607 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:00:13.781047 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:00:13.781047 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 20:00:13.781047 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:00:13.781047 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:00:13.781047 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:00:13.781047 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:00:13.781047 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:00:13.781047 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:00:13.781047 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:00:13.781047 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:00:13.781047 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 20:00:13.781047 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 20:00:13.781047 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 06 20:00:13.781047 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:00:13.781047 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 408/841] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Sep 6 20:00:13 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:00:13.786125 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ec7426-6b7a-4c52-ab22-bd9260770cfd None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:13.833454 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d3d6e4-ffeb-4a80-883f-2250def0d962 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:13.834774 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:13.881872 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:13.892886 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f53fe5-2d69-480b-8574-10ad06c2ed46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:13.893534 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f53fe5-2d69-480b-8574-10ad06c2ed46 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:13.893845 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f53fe5-2d69-480b-8574-10ad06c2ed46 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:13.894120 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f53fe5-2d69-480b-8574-10ad06c2ed46 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:13.903171 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f53fe5-2d69-480b-8574-10ad06c2ed46 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:13.906082 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f53fe5-2d69-480b-8574-10ad06c2ed46 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:13.922676 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-08f53fe5-2d69-480b-8574-10ad06c2ed46 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 06 20:00:13.922676 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:13.922676 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:00:13.922676 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:00:13.922676 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:00:13.922676 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:00:13.922676 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:00:13.922676 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:00:13.922676 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:00:13.922676 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:00:13.922676 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:00:13.922676 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:00:13.922676 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 20:00:13.922676 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 20:00:13.922676 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 20:00:13.922676 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 20:00:13.924576 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:00:13.924576 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:00:13.924576 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 20:00:13.924576 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 20:00:13.924576 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 20:00:13.924576 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 20:00:13.924576 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:00:13.924576 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:00:13.924576 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:00:13.924576 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:00:13.924576 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:00:13.924576 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 20:00:13.924576 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:00:13.924576 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:00:13.924576 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:00:13.924576 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:00:13.924576 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:00:13.924576 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:00:13.925748 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:00:13.925748 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:00:13.925748 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 20:00:13.925748 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 20:00:13.925748 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 20:00:13.925748 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 20:00:13.925748 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 20:00:13.925748 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 20:00:13.925748 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 06 20:00:13.925748 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 20:00:13.926721 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 434/842] 10.222.0.20 () {56 vars in 1117 bytes} [Tue Sep 6 20:00:13 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:00:13.927659 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f53fe5-2d69-480b-8574-10ad06c2ed46 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:13.940791 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ec7426-6b7a-4c52-ab22-bd9260770cfd None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:13.993718 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51e04b3-be71-4c32-979e-f13d9441d4e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:13.994956 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51e04b3-be71-4c32-979e-f13d9441d4e2 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:13.995231 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51e04b3-be71-4c32-979e-f13d9441d4e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:13.995500 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51e04b3-be71-4c32-979e-f13d9441d4e2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:14.011754 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51e04b3-be71-4c32-979e-f13d9441d4e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 20:00:14.019188 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-c51e04b3-be71-4c32-979e-f13d9441d4e2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 20:00:14.019188 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:14.019188 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:00:14.019188 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:00:14.019188 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:00:14.019188 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:00:14.019188 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:00:14.019188 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:00:14.019188 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:00:14.019188 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:00:14.019188 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:00:14.019188 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:00:14.019188 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 20:00:14.019188 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 20:00:14.019188 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 20:00:14.019188 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 20:00:14.021379 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 20:00:14.021379 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 20:00:14.021379 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 20:00:14.021379 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 20:00:14.021379 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:00:14.021379 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:00:14.021379 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:00:14.021379 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:00:14.021379 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:00:14.021379 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:00:14.021379 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:00:14.021379 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 20:00:14.021379 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:00:14.021379 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:00:14.021379 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:00:14.021379 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:00:14.023743 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:00:14.023743 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:00:14.023743 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:00:14.023743 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:00:14.023743 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 20:00:14.023743 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 20:00:14.023743 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 20:00:14.023743 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 20:00:14.023743 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 20:00:14.023743 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 20:00:14.023743 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 20:00:14.023743 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:00:14.023743 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 409/843] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Sep 6 20:00:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:00:14.023743 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51e04b3-be71-4c32-979e-f13d9441d4e2 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:14.073799 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f53fe5-2d69-480b-8574-10ad06c2ed46 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:14.126229 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d4d360-408e-41c9-b0da-2789764b8afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:14.127163 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d4d360-408e-41c9-b0da-2789764b8afd None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:14.127718 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d4d360-408e-41c9-b0da-2789764b8afd None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:14.128058 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d4d360-408e-41c9-b0da-2789764b8afd None admin] PATH_INFO: `/v3/projects` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:14.131538 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d4d360-408e-41c9-b0da-2789764b8afd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:14.131538 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d4d360-408e-41c9-b0da-2789764b8afd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:14.132622 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d4d360-408e-41c9-b0da-2789764b8afd None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:14.151808 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 435/844] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Sep 6 20:00:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:00:14.152747 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d4d360-408e-41c9-b0da-2789764b8afd None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:14.190379 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:14.195119 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51e04b3-be71-4c32-979e-f13d9441d4e2 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:14.238759 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0ba2e8-38ad-4991-b73c-5107ed75c927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:14.239482 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0ba2e8-38ad-4991-b73c-5107ed75c927 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:14.239822 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0ba2e8-38ad-4991-b73c-5107ed75c927 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:14.240349 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0ba2e8-38ad-4991-b73c-5107ed75c927 None admin] PATH_INFO: `/v3/services` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:14.241091 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0ba2e8-38ad-4991-b73c-5107ed75c927 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:14.241467 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0ba2e8-38ad-4991-b73c-5107ed75c927 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:14.242986 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0ba2e8-38ad-4991-b73c-5107ed75c927 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:14.248894 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 410/845] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Sep 6 20:00:14 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:00:14.249891 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0ba2e8-38ad-4991-b73c-5107ed75c927 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:14.256200 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:14.321441 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d4d360-408e-41c9-b0da-2789764b8afd None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:14.368850 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3fc90d-4496-4ecd-8436-8b504c3f58a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:14.369476 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3fc90d-4496-4ecd-8436-8b504c3f58a3 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:14.369791 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3fc90d-4496-4ecd-8436-8b504c3f58a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:14.370060 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3fc90d-4496-4ecd-8436-8b504c3f58a3 None admin] PATH_INFO: `/v3/users` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:14.370818 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3fc90d-4496-4ecd-8436-8b504c3f58a3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:14.371120 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3fc90d-4496-4ecd-8436-8b504c3f58a3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:14.372615 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3fc90d-4496-4ecd-8436-8b504c3f58a3 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:14.395883 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 436/846] 10.222.0.20 () {56 vars in 1149 bytes} [Tue Sep 6 20:00:14 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:00:14.397345 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3fc90d-4496-4ecd-8436-8b504c3f58a3 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:14.429365 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0ba2e8-38ad-4991-b73c-5107ed75c927 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:14.477242 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916fdaab-4858-40f6-9e4a-50cbb1950575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:14.478142 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916fdaab-4858-40f6-9e4a-50cbb1950575 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:14.478530 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916fdaab-4858-40f6-9e4a-50cbb1950575 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:14.479001 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916fdaab-4858-40f6-9e4a-50cbb1950575 None admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:14.479841 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916fdaab-4858-40f6-9e4a-50cbb1950575 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:14.480255 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916fdaab-4858-40f6-9e4a-50cbb1950575 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:14.481738 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916fdaab-4858-40f6-9e4a-50cbb1950575 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:14.496932 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 411/847] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 6 20:00:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:00:14.498699 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916fdaab-4858-40f6-9e4a-50cbb1950575 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:14.535320 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3fc90d-4496-4ecd-8436-8b504c3f58a3 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:14.539366 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:14.547070 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:14.586801 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1242f3-546d-4cc6-b06b-cc36918feed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:14.587408 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1242f3-546d-4cc6-b06b-cc36918feed7 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:14.587762 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1242f3-546d-4cc6-b06b-cc36918feed7 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:14.588147 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1242f3-546d-4cc6-b06b-cc36918feed7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:14.589018 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1242f3-546d-4cc6-b06b-cc36918feed7 None admin] RBAC: Adding query filter params (role.id=e0ab41d8ae22474688b5361bcc75e0f0, scope.project.id=6aef43493bb1420a9e122313d8db7d5a, user.id=5776cbf4f6714b2083c3ebdd6d98cc87) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:14.589479 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1242f3-546d-4cc6-b06b-cc36918feed7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:14.591138 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1242f3-546d-4cc6-b06b-cc36918feed7 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:14.597859 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 437/848] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Sep 6 20:00:14 2022] GET /identity/v3/role_assignments?user.id=5776cbf4f6714b2083c3ebdd6d98cc87&scope.project.id=6aef43493bb1420a9e122313d8db7d5a&role.id=e0ab41d8ae22474688b5361bcc75e0f0 => generated 608 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:00:14.598742 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1242f3-546d-4cc6-b06b-cc36918feed7 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:14.689596 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916fdaab-4858-40f6-9e4a-50cbb1950575 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:14.749679 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd10360-4890-4a29-a10f-9fd8a2925002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:14.750710 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd10360-4890-4a29-a10f-9fd8a2925002 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:14.750866 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd10360-4890-4a29-a10f-9fd8a2925002 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:14.751210 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd10360-4890-4a29-a10f-9fd8a2925002 None admin] PATH_INFO: `/v3/services` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:14.752124 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd10360-4890-4a29-a10f-9fd8a2925002 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:14.752529 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd10360-4890-4a29-a10f-9fd8a2925002 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:14.754312 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd10360-4890-4a29-a10f-9fd8a2925002 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:14.760634 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 412/849] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Sep 6 20:00:14 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:00:14.762131 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd10360-4890-4a29-a10f-9fd8a2925002 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:14.800602 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1242f3-546d-4cc6-b06b-cc36918feed7 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:14.880050 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c212fac-0137-4531-b1a6-6c76a47c9921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:14.882428 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c212fac-0137-4531-b1a6-6c76a47c9921 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:14.882993 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c212fac-0137-4531-b1a6-6c76a47c9921 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:14.883502 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c212fac-0137-4531-b1a6-6c76a47c9921 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:14.962058 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:14.964320 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:14.985067 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd10360-4890-4a29-a10f-9fd8a2925002 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:15.027831 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c212fac-0137-4531-b1a6-6c76a47c9921 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:15.031210 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c212fac-0137-4531-b1a6-6c76a47c9921 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:15.065322 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e12fd7-2a19-4a73-8bfa-1ad81e8188ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:15.065322 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e12fd7-2a19-4a73-8bfa-1ad81e8188ef None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:15.065829 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e12fd7-2a19-4a73-8bfa-1ad81e8188ef None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:15.066029 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e12fd7-2a19-4a73-8bfa-1ad81e8188ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:15.089379 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e12fd7-2a19-4a73-8bfa-1ad81e8188ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:15.091506 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e12fd7-2a19-4a73-8bfa-1ad81e8188ef None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:15.125033 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-24e12fd7-2a19-4a73-8bfa-1ad81e8188ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 20:00:15.125033 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:15.125033 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:00:15.125033 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:00:15.125033 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:00:15.125033 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:00:15.125033 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:00:15.125033 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:00:15.125033 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:00:15.125033 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:00:15.125033 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:00:15.125033 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:00:15.125033 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 20:00:15.125033 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 20:00:15.125033 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 20:00:15.125033 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 20:00:15.126325 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:00:15.126325 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:00:15.126325 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:00:15.126325 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:00:15.126325 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:00:15.126325 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:00:15.126325 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:00:15.126325 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 20:00:15.126325 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:00:15.126325 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:00:15.126325 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:00:15.126325 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:00:15.126325 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:00:15.126325 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:00:15.126325 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:00:15.126325 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:00:15.126325 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 20:00:15.127322 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 20:00:15.127322 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 20:00:15.127322 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:00:15.127957 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 413/850] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 20:00:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 366 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:00:15.135010 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e12fd7-2a19-4a73-8bfa-1ad81e8188ef None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:15.197251 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 438/851] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 6 20:00:14 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 599 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 20:00:15.204574 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c212fac-0137-4531-b1a6-6c76a47c9921 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:15.284357 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e12fd7-2a19-4a73-8bfa-1ad81e8188ef None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:15.356330 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:15.357938 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d24f56e-9c6b-498b-a2ec-43c9b5f84459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:15.358785 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d24f56e-9c6b-498b-a2ec-43c9b5f84459 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:15.359280 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d24f56e-9c6b-498b-a2ec-43c9b5f84459 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:15.359805 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d24f56e-9c6b-498b-a2ec-43c9b5f84459 None admin] PATH_INFO: `/v3/domains` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:15.360823 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d24f56e-9c6b-498b-a2ec-43c9b5f84459 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:15.361339 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d24f56e-9c6b-498b-a2ec-43c9b5f84459 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:15.363038 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d24f56e-9c6b-498b-a2ec-43c9b5f84459 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:15.379329 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 414/852] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 6 20:00:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:00:15.381112 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:15.390704 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d24f56e-9c6b-498b-a2ec-43c9b5f84459 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:15.419091 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c212fac-0137-4531-b1a6-6c76a47c9921 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:15.474141 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2efa8fa-836a-4dbd-855f-f0c07b8cda90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:15.475220 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2efa8fa-836a-4dbd-855f-f0c07b8cda90 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:15.475705 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2efa8fa-836a-4dbd-855f-f0c07b8cda90 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:15.476181 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2efa8fa-836a-4dbd-855f-f0c07b8cda90 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:15.501403 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2efa8fa-836a-4dbd-855f-f0c07b8cda90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:15.504618 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2efa8fa-836a-4dbd-855f-f0c07b8cda90 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:15.536251 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-e2efa8fa-836a-4dbd-855f-f0c07b8cda90 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 20:00:15.536251 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:15.536251 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:00:15.536251 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:00:15.536251 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:00:15.536251 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:00:15.536251 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:00:15.536251 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:00:15.536251 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:00:15.536251 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:00:15.536251 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:00:15.536251 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:00:15.536251 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 20:00:15.536251 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 20:00:15.536251 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 20:00:15.536251 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 20:00:15.538497 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:00:15.538497 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:00:15.538497 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:00:15.538497 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:00:15.538497 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:00:15.538497 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:00:15.538497 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:00:15.538497 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 20:00:15.538497 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:00:15.538497 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:00:15.538497 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:00:15.538497 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:00:15.538497 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:00:15.538497 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:00:15.538497 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:00:15.538497 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:00:15.538497 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 20:00:15.540523 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 20:00:15.540523 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 20:00:15.540523 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 20:00:15.546552 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 439/853] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 20:00:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:00:15.552142 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2efa8fa-836a-4dbd-855f-f0c07b8cda90 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:15.566765 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d24f56e-9c6b-498b-a2ec-43c9b5f84459 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:15.610739 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1465c78a-22ff-4b13-9c54-5bf3cb6fdac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:15.611471 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1465c78a-22ff-4b13-9c54-5bf3cb6fdac8 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:15.611798 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1465c78a-22ff-4b13-9c54-5bf3cb6fdac8 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:15.612103 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1465c78a-22ff-4b13-9c54-5bf3cb6fdac8 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:15.620943 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1465c78a-22ff-4b13-9c54-5bf3cb6fdac8 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:15.622617 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1465c78a-22ff-4b13-9c54-5bf3cb6fdac8 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:15.637163 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-1465c78a-22ff-4b13-9c54-5bf3cb6fdac8 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 06 20:00:15.637163 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:15.637163 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:00:15.637163 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:00:15.637163 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:00:15.637163 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:00:15.637163 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:00:15.637163 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:00:15.637163 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:00:15.637163 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:00:15.637163 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:00:15.637163 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:00:15.637163 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 20:00:15.637163 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 20:00:15.637163 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 20:00:15.637163 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 20:00:15.639677 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:00:15.639677 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:00:15.639677 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 20:00:15.639677 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 20:00:15.639677 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 20:00:15.639677 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 20:00:15.639677 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:00:15.639677 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:00:15.639677 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:00:15.639677 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:00:15.639677 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:00:15.639677 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 20:00:15.639677 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:00:15.639677 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:00:15.639677 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:00:15.639677 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:00:15.639677 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:00:15.639677 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:00:15.641875 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:00:15.641875 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:00:15.641875 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 20:00:15.641875 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 20:00:15.641875 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 20:00:15.641875 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 20:00:15.641875 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 20:00:15.641875 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 20:00:15.641875 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 06 20:00:15.641875 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:00:15.641875 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 415/854] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Sep 6 20:00:15 2022] GET /identity/v3/users/swift => generated 83 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:00:15.646089 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1465c78a-22ff-4b13-9c54-5bf3cb6fdac8 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:15.755943 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2efa8fa-836a-4dbd-855f-f0c07b8cda90 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:15.805589 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:15.810997 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:15.831935 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daeb65d-e906-4af6-ba26-075e09088466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:15.832994 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daeb65d-e906-4af6-ba26-075e09088466 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:15.833287 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daeb65d-e906-4af6-ba26-075e09088466 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:15.833683 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daeb65d-e906-4af6-ba26-075e09088466 None admin] PATH_INFO: `/v3/domains` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:15.834522 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daeb65d-e906-4af6-ba26-075e09088466 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:15.834829 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daeb65d-e906-4af6-ba26-075e09088466 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:15.836346 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daeb65d-e906-4af6-ba26-075e09088466 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:15.857713 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 440/855] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 6 20:00:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 20:00:15.865002 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daeb65d-e906-4af6-ba26-075e09088466 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:15.895716 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1465c78a-22ff-4b13-9c54-5bf3cb6fdac8 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:15.950479 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567611c8-c60f-4eea-af8a-384a9c11b21a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:15.951190 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567611c8-c60f-4eea-af8a-384a9c11b21a None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:15.951478 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567611c8-c60f-4eea-af8a-384a9c11b21a None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:15.951822 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567611c8-c60f-4eea-af8a-384a9c11b21a None admin] PATH_INFO: `/v3/users` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:15.952509 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567611c8-c60f-4eea-af8a-384a9c11b21a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:15.952825 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567611c8-c60f-4eea-af8a-384a9c11b21a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:15.954696 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567611c8-c60f-4eea-af8a-384a9c11b21a None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:15.993101 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 416/856] 10.222.0.20 () {56 vars in 1135 bytes} [Tue Sep 6 20:00:15 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 20:00:16.000430 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567611c8-c60f-4eea-af8a-384a9c11b21a None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:16.079003 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daeb65d-e906-4af6-ba26-075e09088466 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:16.138013 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567611c8-c60f-4eea-af8a-384a9c11b21a None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:16.155707 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b2e47e-1b1f-4af9-9bff-a52718c5e3c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:16.157270 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b2e47e-1b1f-4af9-9bff-a52718c5e3c1 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:16.158111 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b2e47e-1b1f-4af9-9bff-a52718c5e3c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:16.158873 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b2e47e-1b1f-4af9-9bff-a52718c5e3c1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:16.160944 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:16.174733 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b2e47e-1b1f-4af9-9bff-a52718c5e3c1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:16.178339 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b2e47e-1b1f-4af9-9bff-a52718c5e3c1 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:16.184147 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43beea7-fedf-491d-897e-950b56ad13d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:16.184769 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43beea7-fedf-491d-897e-950b56ad13d6 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:16.185054 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43beea7-fedf-491d-897e-950b56ad13d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:16.185407 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43beea7-fedf-491d-897e-950b56ad13d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:16.206557 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-48b2e47e-1b1f-4af9-9bff-a52718c5e3c1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 20:00:16.206557 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:16.206557 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:00:16.206557 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:00:16.206557 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:00:16.206557 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:00:16.206557 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:00:16.206557 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:00:16.206557 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:00:16.206557 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:00:16.206557 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:00:16.206557 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:00:16.206557 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 20:00:16.206557 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 20:00:16.206557 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 20:00:16.208972 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 20:00:16.208972 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:00:16.208972 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:00:16.208972 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:00:16.208972 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:00:16.208972 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:00:16.208972 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:00:16.208972 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:00:16.208972 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 20:00:16.208972 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:00:16.208972 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:00:16.208972 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:00:16.208972 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:00:16.208972 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:00:16.208972 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:00:16.208972 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:00:16.208972 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:00:16.210809 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 20:00:16.210809 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 20:00:16.210809 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 20:00:16.210809 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 20:00:16.210809 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 20:00:16.210809 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 20:00:16.210809 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 20:00:16.210809 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 20:00:16.222237 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 441/857] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 6 20:00:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 359 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:00:16.225955 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b2e47e-1b1f-4af9-9bff-a52718c5e3c1 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:16.272034 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43beea7-fedf-491d-897e-950b56ad13d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:16.273750 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43beea7-fedf-491d-897e-950b56ad13d6 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:16.321196 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:16.400203 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 417/858] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 6 20:00:15 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 20:00:16.407539 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43beea7-fedf-491d-897e-950b56ad13d6 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:16.447443 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:16.460662 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b2e47e-1b1f-4af9-9bff-a52718c5e3c1 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:16.543747 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f03baf-8d2a-49bb-9e45-5d5c1db64b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:16.545379 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f03baf-8d2a-49bb-9e45-5d5c1db64b7d None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:16.546344 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f03baf-8d2a-49bb-9e45-5d5c1db64b7d None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:16.547223 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f03baf-8d2a-49bb-9e45-5d5c1db64b7d None admin] PATH_INFO: `/v3/projects` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:16.548823 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f03baf-8d2a-49bb-9e45-5d5c1db64b7d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:16.549771 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f03baf-8d2a-49bb-9e45-5d5c1db64b7d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:16.553437 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f03baf-8d2a-49bb-9e45-5d5c1db64b7d None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:16.593660 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 442/859] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Sep 6 20:00:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 20:00:16.601373 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f03baf-8d2a-49bb-9e45-5d5c1db64b7d None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:16.644472 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43beea7-fedf-491d-897e-950b56ad13d6 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:16.728495 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acddaba-55a8-4d07-83c7-0ed728f8b4f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:16.729671 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acddaba-55a8-4d07-83c7-0ed728f8b4f6 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:16.730160 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acddaba-55a8-4d07-83c7-0ed728f8b4f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:16.730626 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acddaba-55a8-4d07-83c7-0ed728f8b4f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:16.755717 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acddaba-55a8-4d07-83c7-0ed728f8b4f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:16.759064 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acddaba-55a8-4d07-83c7-0ed728f8b4f6 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:16.782962 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-0acddaba-55a8-4d07-83c7-0ed728f8b4f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 20:00:16.782962 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:16.782962 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:00:16.782962 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:00:16.782962 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:00:16.782962 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:00:16.782962 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:00:16.782962 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:00:16.782962 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:00:16.782962 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:00:16.782962 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:00:16.782962 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:00:16.782962 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 20:00:16.782962 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 20:00:16.782962 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 20:00:16.782962 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 20:00:16.784880 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:00:16.784880 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:00:16.784880 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:00:16.784880 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:00:16.784880 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:00:16.784880 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:00:16.784880 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:00:16.784880 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 20:00:16.784880 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:00:16.784880 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:00:16.784880 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:00:16.784880 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:00:16.784880 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:00:16.784880 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:00:16.784880 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:00:16.784880 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:00:16.784880 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 20:00:16.786680 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 20:00:16.786680 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 20:00:16.786680 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:00:16.786680 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 418/860] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 20:00:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 379 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:00:16.795920 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acddaba-55a8-4d07-83c7-0ed728f8b4f6 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:16.844213 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f03baf-8d2a-49bb-9e45-5d5c1db64b7d None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:16.848768 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:16.918202 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe6755d-d34e-4921-b6fd-bee33e86b348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:16.919143 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe6755d-d34e-4921-b6fd-bee33e86b348 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:16.920334 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe6755d-d34e-4921-b6fd-bee33e86b348 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:16.920619 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe6755d-d34e-4921-b6fd-bee33e86b348 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:16.921435 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe6755d-d34e-4921-b6fd-bee33e86b348 None admin] RBAC: Adding query filter params (role.id=b950bf73c93e4f7ebed923ff14e39faa, scope.project.id=6aef43493bb1420a9e122313d8db7d5a, user.id=1a52117df70447dba9bbee35d083e44f) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:16.921825 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe6755d-d34e-4921-b6fd-bee33e86b348 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:16.923430 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe6755d-d34e-4921-b6fd-bee33e86b348 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:16.937670 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 443/861] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Sep 6 20:00:16 2022] GET /identity/v3/role_assignments?user.id=1a52117df70447dba9bbee35d083e44f&scope.project.id=6aef43493bb1420a9e122313d8db7d5a&role.id=b950bf73c93e4f7ebed923ff14e39faa => generated 608 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 20:00:16.978214 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:16.998269 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acddaba-55a8-4d07-83c7-0ed728f8b4f6 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:17.051398 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df85f060-e02e-44c4-a6b3-f2693a337759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:17.052178 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df85f060-e02e-44c4-a6b3-f2693a337759 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:17.052637 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df85f060-e02e-44c4-a6b3-f2693a337759 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:17.053109 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df85f060-e02e-44c4-a6b3-f2693a337759 None admin] PATH_INFO: `/v3/domains` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:17.054182 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df85f060-e02e-44c4-a6b3-f2693a337759 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:17.054520 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df85f060-e02e-44c4-a6b3-f2693a337759 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:17.056578 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df85f060-e02e-44c4-a6b3-f2693a337759 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:17.075337 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 419/862] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 6 20:00:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:00:17.081859 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe6755d-d34e-4921-b6fd-bee33e86b348 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:17.298789 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe6755d-d34e-4921-b6fd-bee33e86b348 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:17.380914 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45765718-144c-4f2f-9131-a8cff73147cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:17.382526 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45765718-144c-4f2f-9131-a8cff73147cc None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:17.383244 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45765718-144c-4f2f-9131-a8cff73147cc None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:17.383948 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45765718-144c-4f2f-9131-a8cff73147cc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:17.396207 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45765718-144c-4f2f-9131-a8cff73147cc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:17.398285 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45765718-144c-4f2f-9131-a8cff73147cc None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:17.420004 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-45765718-144c-4f2f-9131-a8cff73147cc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 20:00:17.420004 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:17.420004 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:00:17.420004 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:00:17.420004 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:00:17.420004 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:00:17.420004 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:00:17.420004 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:00:17.420004 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:00:17.420004 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:00:17.420004 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:00:17.420004 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:00:17.420004 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 20:00:17.420004 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 20:00:17.420004 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 20:00:17.421604 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 20:00:17.421604 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:00:17.421604 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:00:17.421604 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:00:17.421604 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:00:17.421604 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:00:17.421604 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:00:17.421604 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:00:17.421604 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 20:00:17.421604 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:00:17.421604 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:00:17.421604 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:00:17.421604 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:00:17.421604 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:00:17.421604 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:00:17.421604 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:00:17.421604 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:00:17.423215 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 20:00:17.423215 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 20:00:17.423215 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 20:00:17.423215 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 20:00:17.423215 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 20:00:17.423215 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 20:00:17.423215 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 20:00:17.423215 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 20:00:17.423215 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 444/863] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 6 20:00:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:00:17.432561 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df85f060-e02e-44c4-a6b3-f2693a337759 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:17.474162 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742a46f3-fcbc-437c-9cea-5d64557d2e7f None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:17.474650 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742a46f3-fcbc-437c-9cea-5d64557d2e7f None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:17.475568 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742a46f3-fcbc-437c-9cea-5d64557d2e7f None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:17.479512 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 445/864] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:00:17 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:00:17.487147 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7b2cb3-c40e-4bd1-af6e-8fb43000a2f4 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:17.488444 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7b2cb3-c40e-4bd1-af6e-8fb43000a2f4 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:17.489079 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7b2cb3-c40e-4bd1-af6e-8fb43000a2f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:17.531991 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-9b7b2cb3-c40e-4bd1-af6e-8fb43000a2f4 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:00:17.534308 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-9b7b2cb3-c40e-4bd1-af6e-8fb43000a2f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:00:17.594149 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:17.613865 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df85f060-e02e-44c4-a6b3-f2693a337759 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:17.668739 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:17.678678 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bf2906-f515-4f76-bbeb-3dcb36bca2ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:17.679301 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bf2906-f515-4f76-bbeb-3dcb36bca2ae None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:17.680102 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bf2906-f515-4f76-bbeb-3dcb36bca2ae None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:17.680481 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bf2906-f515-4f76-bbeb-3dcb36bca2ae None admin] PATH_INFO: `/v3/projects` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:17.681528 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bf2906-f515-4f76-bbeb-3dcb36bca2ae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:17.682099 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bf2906-f515-4f76-bbeb-3dcb36bca2ae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:17.683792 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bf2906-f515-4f76-bbeb-3dcb36bca2ae None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:17.705957 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 420/865] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Sep 6 20:00:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:00:17.708657 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bf2906-f515-4f76-bbeb-3dcb36bca2ae None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:17.738033 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 446/866] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:00:17 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 20:00:17.915239 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f5bbeb-4a57-472d-a7e7-631cded49593 None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:17.915521 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f5bbeb-4a57-472d-a7e7-631cded49593 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:17.916020 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f5bbeb-4a57-472d-a7e7-631cded49593 None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:17.918258 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 447/867] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:00:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:00:17.923306 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bf2906-f515-4f76-bbeb-3dcb36bca2ae None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:17.924484 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268b8ad1-7892-4dcf-8bd5-9e1b3b2f0ed6 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:17.924844 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268b8ad1-7892-4dcf-8bd5-9e1b3b2f0ed6 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:17.925193 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268b8ad1-7892-4dcf-8bd5-9e1b3b2f0ed6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:17.947726 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-268b8ad1-7892-4dcf-8bd5-9e1b3b2f0ed6 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:00:17.949305 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-268b8ad1-7892-4dcf-8bd5-9e1b3b2f0ed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:00:17.970032 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd26900-8e5d-4924-9219-dd66f3b07b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:17.970919 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd26900-8e5d-4924-9219-dd66f3b07b0b None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:17.971344 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd26900-8e5d-4924-9219-dd66f3b07b0b None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:17.971809 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd26900-8e5d-4924-9219-dd66f3b07b0b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:17.972703 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd26900-8e5d-4924-9219-dd66f3b07b0b None admin] RBAC: Adding query filter params (role.id=e0ab41d8ae22474688b5361bcc75e0f0, scope.project.id=6aef43493bb1420a9e122313d8db7d5a, user.id=2b72516236da47e292d78de201b0552b) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:17.973014 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd26900-8e5d-4924-9219-dd66f3b07b0b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:17.974584 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd26900-8e5d-4924-9219-dd66f3b07b0b None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:17.988262 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 421/868] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Sep 6 20:00:17 2022] GET /identity/v3/role_assignments?user.id=2b72516236da47e292d78de201b0552b&scope.project.id=6aef43493bb1420a9e122313d8db7d5a&role.id=e0ab41d8ae22474688b5361bcc75e0f0 => generated 608 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 20:00:18.047927 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 448/869] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:00:17 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 20:00:18.141566 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd26900-8e5d-4924-9219-dd66f3b07b0b None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:18.241164 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:18.328480 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd26900-8e5d-4924-9219-dd66f3b07b0b None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:18.394693 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5222530e-a58d-4216-b2c4-62c5b7542084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:18.395711 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5222530e-a58d-4216-b2c4-62c5b7542084 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:18.396187 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5222530e-a58d-4216-b2c4-62c5b7542084 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:18.396644 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5222530e-a58d-4216-b2c4-62c5b7542084 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:18.397599 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5222530e-a58d-4216-b2c4-62c5b7542084 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:18.399901 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5222530e-a58d-4216-b2c4-62c5b7542084 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:18.415831 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-5222530e-a58d-4216-b2c4-62c5b7542084 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 06 20:00:18.415831 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:18.415831 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 20:00:18.415831 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 20:00:18.415831 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 20:00:18.415831 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 20:00:18.415831 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 20:00:18.415831 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 20:00:18.415831 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 06 20:00:18.415831 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:00:18.415831 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 20:00:18.415831 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:00:18.415831 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:18.415831 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:00:18.415831 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:00:18.415831 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:00:18.417693 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:00:18.417693 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:00:18.417693 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:00:18.417693 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:00:18.417693 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:00:18.417693 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:00:18.417693 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:00:18.417693 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 20:00:18.417693 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 20:00:18.417693 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 20:00:18.417693 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 20:00:18.417693 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:00:18.417693 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:00:18.417693 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:00:18.417693 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:00:18.417693 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:00:18.417693 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:00:18.420652 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:00:18.420652 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 20:00:18.420652 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:00:18.420652 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:00:18.420652 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:00:18.420652 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:00:18.420652 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:00:18.420652 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:00:18.420652 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:00:18.420652 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:00:18.420652 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 20:00:18.420652 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 20:00:18.420652 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 06 20:00:18.420652 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:00:18.420652 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 422/870] 10.222.0.20 () {56 vars in 1132 bytes} [Tue Sep 6 20:00:18 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:00:18.432580 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268b8ad1-7892-4dcf-8bd5-9e1b3b2f0ed6 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:18.473177 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5222530e-a58d-4216-b2c4-62c5b7542084 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:18.577666 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:18.594628 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268b8ad1-7892-4dcf-8bd5-9e1b3b2f0ed6 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:18.651225 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb7c8f7-4769-4d82-a616-8a027e6d3a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:18.651940 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb7c8f7-4769-4d82-a616-8a027e6d3a3a None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:18.652224 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb7c8f7-4769-4d82-a616-8a027e6d3a3a None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:18.652497 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb7c8f7-4769-4d82-a616-8a027e6d3a3a None admin] PATH_INFO: `/v3/services` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:18.653543 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb7c8f7-4769-4d82-a616-8a027e6d3a3a None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:18.654978 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb7c8f7-4769-4d82-a616-8a027e6d3a3a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:18.656453 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb7c8f7-4769-4d82-a616-8a027e6d3a3a None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:18.663503 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 449/871] 10.222.0.20 () {56 vars in 1126 bytes} [Tue Sep 6 20:00:18 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:00:18.668971 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb7c8f7-4769-4d82-a616-8a027e6d3a3a None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:18.696142 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5222530e-a58d-4216-b2c4-62c5b7542084 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:18.717589 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:18.739886 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87367f0d-b315-4286-9436-cfe9ac3f12b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:18.740542 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87367f0d-b315-4286-9436-cfe9ac3f12b9 None admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:18.740800 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87367f0d-b315-4286-9436-cfe9ac3f12b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:18.741217 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87367f0d-b315-4286-9436-cfe9ac3f12b9 None admin] PATH_INFO: `/v3/services` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:18.742014 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87367f0d-b315-4286-9436-cfe9ac3f12b9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:18.743512 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87367f0d-b315-4286-9436-cfe9ac3f12b9 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:18.769992 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 423/872] 10.222.0.20 () {60 vars in 1141 bytes} [Tue Sep 6 20:00:18 2022] POST /identity/v3/services => generated 251 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:00:18.820109 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb7c8f7-4769-4d82-a616-8a027e6d3a3a None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:18.877590 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febde422-908d-4eff-ac3c-cc019e842524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:18.878447 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febde422-908d-4eff-ac3c-cc019e842524 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:18.878759 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febde422-908d-4eff-ac3c-cc019e842524 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:18.879051 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febde422-908d-4eff-ac3c-cc019e842524 None admin] PATH_INFO: `/v3/services` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:18.880906 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febde422-908d-4eff-ac3c-cc019e842524 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:18.880906 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febde422-908d-4eff-ac3c-cc019e842524 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:18.885648 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febde422-908d-4eff-ac3c-cc019e842524 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:18.893806 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 450/873] 10.222.0.20 () {56 vars in 1126 bytes} [Tue Sep 6 20:00:18 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:00:20.531167 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8046e193-8109-4866-987f-2146b4e33252 None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:20.531911 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8046e193-8109-4866-987f-2146b4e33252 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:20.531911 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8046e193-8109-4866-987f-2146b4e33252 None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:20.533818 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 424/874] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:00:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:00:20.539862 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e71edb-76d5-45dc-989c-858c6f7a0b8c None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:20.540418 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e71edb-76d5-45dc-989c-858c6f7a0b8c None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:20.540651 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e71edb-76d5-45dc-989c-858c6f7a0b8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:20.573165 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-15e71edb-76d5-45dc-989c-858c6f7a0b8c None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:00:20.575242 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-15e71edb-76d5-45dc-989c-858c6f7a0b8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:00:20.580856 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:20.713532 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 451/875] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:00:20 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 20:00:20.929215 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d8b8f3-755a-4f8c-b40a-338fa5b6ee3d None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:20.929560 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d8b8f3-755a-4f8c-b40a-338fa5b6ee3d None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:20.929818 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d8b8f3-755a-4f8c-b40a-338fa5b6ee3d None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:20.931832 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 425/876] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:00:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:00:20.939198 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb54f4b-5062-4222-93b5-21b108f7dfad None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:20.939545 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb54f4b-5062-4222-93b5-21b108f7dfad None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:20.939756 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb54f4b-5062-4222-93b5-21b108f7dfad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:20.961855 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-0cb54f4b-5062-4222-93b5-21b108f7dfad None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:00:20.963120 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-0cb54f4b-5062-4222-93b5-21b108f7dfad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:00:21.022602 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d8b8f3-755a-4f8c-b40a-338fa5b6ee3d None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:21.065104 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 452/877] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:00:20 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 20:00:21.222546 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d8b8f3-755a-4f8c-b40a-338fa5b6ee3d None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:21.270266 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13dfdcc-7793-48cf-88a5-485c864746e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:21.271357 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13dfdcc-7793-48cf-88a5-485c864746e5 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:21.271670 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13dfdcc-7793-48cf-88a5-485c864746e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:21.271972 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13dfdcc-7793-48cf-88a5-485c864746e5 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:21.272672 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13dfdcc-7793-48cf-88a5-485c864746e5 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:21.274502 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13dfdcc-7793-48cf-88a5-485c864746e5 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:21.286963 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-e13dfdcc-7793-48cf-88a5-485c864746e5 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 20:00:21.286963 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:21.286963 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 20:00:21.286963 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 20:00:21.286963 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 20:00:21.286963 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 20:00:21.286963 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 20:00:21.286963 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 20:00:21.286963 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 20:00:21.286963 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:00:21.286963 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 20:00:21.286963 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:00:21.286963 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:21.286963 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:00:21.286963 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:00:21.286963 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:00:21.286963 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:00:21.288933 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:00:21.288933 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:00:21.288933 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:00:21.288933 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:00:21.288933 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:00:21.288933 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:00:21.288933 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 20:00:21.288933 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 20:00:21.288933 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 20:00:21.288933 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 20:00:21.288933 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:00:21.288933 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:00:21.288933 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:00:21.288933 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:00:21.288933 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:00:21.288933 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:00:21.290689 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:00:21.290689 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 20:00:21.290689 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:00:21.290689 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:00:21.290689 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:00:21.290689 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:00:21.290689 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:00:21.290689 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:00:21.290689 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:00:21.290689 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:00:21.290689 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 20:00:21.290689 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 20:00:21.290689 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 20:00:21.290689 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:00:21.293656 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 426/878] 10.222.0.20 () {56 vars in 1126 bytes} [Tue Sep 6 20:00:21 2022] GET /identity/v3/services/object-store => generated 93 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:00:21.295669 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb54f4b-5062-4222-93b5-21b108f7dfad None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:21.364855 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad356032-90a9-44fa-988f-08aa4360b912 None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:21.365367 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad356032-90a9-44fa-988f-08aa4360b912 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:21.365973 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad356032-90a9-44fa-988f-08aa4360b912 None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:21.368677 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 427/879] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:00:21 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:00:21.370133 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad356032-90a9-44fa-988f-08aa4360b912 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:21.393442 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:21.443774 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb54f4b-5062-4222-93b5-21b108f7dfad None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:21.463205 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:21.505232 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a05113c-5302-4697-a7b9-8e5129b2c796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:21.505915 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a05113c-5302-4697-a7b9-8e5129b2c796 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:21.506238 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a05113c-5302-4697-a7b9-8e5129b2c796 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:21.506564 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a05113c-5302-4697-a7b9-8e5129b2c796 None admin] PATH_INFO: `/v3/services` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:21.507296 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a05113c-5302-4697-a7b9-8e5129b2c796 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:21.507662 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a05113c-5302-4697-a7b9-8e5129b2c796 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:21.509516 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a05113c-5302-4697-a7b9-8e5129b2c796 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:21.517308 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 453/880] 10.222.0.20 () {56 vars in 1122 bytes} [Tue Sep 6 20:00:21 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:00:21.519529 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebc5f9f-102c-418d-a2d5-646aa64d9ab0 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:21.520016 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebc5f9f-102c-418d-a2d5-646aa64d9ab0 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:21.520496 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebc5f9f-102c-418d-a2d5-646aa64d9ab0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:21.544710 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-3ebc5f9f-102c-418d-a2d5-646aa64d9ab0 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:00:21.546042 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-3ebc5f9f-102c-418d-a2d5-646aa64d9ab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:00:21.606388 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad356032-90a9-44fa-988f-08aa4360b912 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:21.677759 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837903f6-30c5-42ee-8bdb-35bdc510f4d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:21.678820 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837903f6-30c5-42ee-8bdb-35bdc510f4d9 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:21.679170 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837903f6-30c5-42ee-8bdb-35bdc510f4d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:21.679591 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837903f6-30c5-42ee-8bdb-35bdc510f4d9 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:21.683719 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 454/881] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:00:21 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 20:00:21.687193 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebc5f9f-102c-418d-a2d5-646aa64d9ab0 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:21.702129 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837903f6-30c5-42ee-8bdb-35bdc510f4d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 20:00:21.711906 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-837903f6-30c5-42ee-8bdb-35bdc510f4d9 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 06 20:00:21.711906 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:21.711906 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:00:21.711906 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:00:21.711906 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:00:21.711906 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:00:21.711906 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:00:21.711906 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:00:21.711906 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:00:21.711906 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:00:21.711906 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:00:21.711906 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:00:21.711906 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 20:00:21.711906 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 20:00:21.711906 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 20:00:21.711906 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 20:00:21.713822 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 20:00:21.713822 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 20:00:21.713822 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 20:00:21.713822 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 20:00:21.713822 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:00:21.713822 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:00:21.713822 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:00:21.713822 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:00:21.713822 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:00:21.713822 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:00:21.713822 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:00:21.713822 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 20:00:21.713822 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:00:21.713822 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:00:21.713822 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:00:21.713822 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:00:21.715495 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:00:21.715495 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:00:21.715495 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:00:21.715495 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:00:21.715495 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 20:00:21.715495 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 20:00:21.715495 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 20:00:21.715495 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 20:00:21.715495 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 20:00:21.715495 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 20:00:21.715495 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 06 20:00:21.715495 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:00:21.715495 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 428/882] 10.222.0.20 () {56 vars in 1120 bytes} [Tue Sep 6 20:00:21 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:00:21.723656 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837903f6-30c5-42ee-8bdb-35bdc510f4d9 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:21.845220 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:21.938774 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebc5f9f-102c-418d-a2d5-646aa64d9ab0 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:21.980854 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837903f6-30c5-42ee-8bdb-35bdc510f4d9 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:21.985680 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:22.026666 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f76c77-4411-4c46-9438-9e288b4796fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:22.027891 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f76c77-4411-4c46-9438-9e288b4796fe None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:22.028521 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f76c77-4411-4c46-9438-9e288b4796fe None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:22.029014 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f76c77-4411-4c46-9438-9e288b4796fe None admin] PATH_INFO: `/v3/services` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:22.030383 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f76c77-4411-4c46-9438-9e288b4796fe None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:22.031120 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f76c77-4411-4c46-9438-9e288b4796fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:22.035354 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f76c77-4411-4c46-9438-9e288b4796fe None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:22.047942 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 455/883] 10.222.0.20 () {56 vars in 1122 bytes} [Tue Sep 6 20:00:21 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:00:22.063333 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323a58cc-6853-4b78-89a5-f0ae34ab7639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:22.064373 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323a58cc-6853-4b78-89a5-f0ae34ab7639 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:22.064845 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323a58cc-6853-4b78-89a5-f0ae34ab7639 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:22.065305 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323a58cc-6853-4b78-89a5-f0ae34ab7639 None admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:22.066545 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323a58cc-6853-4b78-89a5-f0ae34ab7639 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:22.067017 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323a58cc-6853-4b78-89a5-f0ae34ab7639 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:22.069346 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323a58cc-6853-4b78-89a5-f0ae34ab7639 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:22.087596 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 429/884] 10.222.0.20 () {56 vars in 1115 bytes} [Tue Sep 6 20:00:21 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:00:22.095654 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f76c77-4411-4c46-9438-9e288b4796fe None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:22.220843 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323a58cc-6853-4b78-89a5-f0ae34ab7639 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:22.309713 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f76c77-4411-4c46-9438-9e288b4796fe None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:22.372779 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:22.384180 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8d4fca-07a5-43f4-9e37-6cc35f8fe2b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:22.385339 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8d4fca-07a5-43f4-9e37-6cc35f8fe2b1 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:22.386561 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8d4fca-07a5-43f4-9e37-6cc35f8fe2b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:22.386972 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8d4fca-07a5-43f4-9e37-6cc35f8fe2b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:22.409142 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8d4fca-07a5-43f4-9e37-6cc35f8fe2b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:22.412116 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8d4fca-07a5-43f4-9e37-6cc35f8fe2b1 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:22.422859 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323a58cc-6853-4b78-89a5-f0ae34ab7639 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:22.444686 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-8d8d4fca-07a5-43f4-9e37-6cc35f8fe2b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 20:00:22.444686 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:22.444686 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:00:22.444686 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:00:22.444686 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:00:22.444686 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:00:22.444686 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:00:22.444686 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:00:22.444686 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:00:22.444686 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:00:22.444686 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:00:22.444686 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:00:22.444686 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 20:00:22.444686 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 20:00:22.444686 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 20:00:22.444686 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 20:00:22.446705 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:00:22.446705 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:00:22.446705 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:00:22.446705 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:00:22.446705 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:00:22.446705 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:00:22.446705 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:00:22.446705 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 20:00:22.446705 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:00:22.446705 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:00:22.446705 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:00:22.446705 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:00:22.446705 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:00:22.446705 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:00:22.446705 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:00:22.446705 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:00:22.446705 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 20:00:22.449334 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 20:00:22.449334 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 20:00:22.449334 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 20:00:22.449334 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 456/885] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 20:00:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 352 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:00:22.449334 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc469a9-c9c1-456c-bfc9-4c1db53fd2d8 None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:22.449334 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc469a9-c9c1-456c-bfc9-4c1db53fd2d8 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:22.450681 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc469a9-c9c1-456c-bfc9-4c1db53fd2d8 None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:22.455623 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 457/886] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:00:22 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:00:22.457060 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc469a9-c9c1-456c-bfc9-4c1db53fd2d8 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:22.504476 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fe9400-951c-4510-9c53-4460e39e566c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:22.505754 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fe9400-951c-4510-9c53-4460e39e566c None admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:22.506376 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fe9400-951c-4510-9c53-4460e39e566c None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:22.506987 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fe9400-951c-4510-9c53-4460e39e566c None admin] PATH_INFO: `/v3/services` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:22.508338 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fe9400-951c-4510-9c53-4460e39e566c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:22.511522 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fe9400-951c-4510-9c53-4460e39e566c None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:22.546279 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 430/887] 10.222.0.20 () {60 vars in 1141 bytes} [Tue Sep 6 20:00:22 2022] POST /identity/v3/services => generated 269 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 20:00:22.547866 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2472141-c171-4d6e-8776-8629bd0a24d5 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:22.548318 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2472141-c171-4d6e-8776-8629bd0a24d5 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:22.548760 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2472141-c171-4d6e-8776-8629bd0a24d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:22.584169 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-a2472141-c171-4d6e-8776-8629bd0a24d5 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:00:22.586518 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-a2472141-c171-4d6e-8776-8629bd0a24d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:00:22.625590 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:22.698161 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc469a9-c9c1-456c-bfc9-4c1db53fd2d8 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:22.746181 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 431/888] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:00:22 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 20:00:22.773083 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7dd3f4-7d2b-41e6-a63d-21c117f4a442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:22.774158 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7dd3f4-7d2b-41e6-a63d-21c117f4a442 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:22.774718 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7dd3f4-7d2b-41e6-a63d-21c117f4a442 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:22.775173 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7dd3f4-7d2b-41e6-a63d-21c117f4a442 None admin] PATH_INFO: `/v3/domains` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:22.776464 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7dd3f4-7d2b-41e6-a63d-21c117f4a442 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:22.777054 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7dd3f4-7d2b-41e6-a63d-21c117f4a442 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:22.779999 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7dd3f4-7d2b-41e6-a63d-21c117f4a442 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:22.811314 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 458/889] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 6 20:00:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:00:22.819991 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2472141-c171-4d6e-8776-8629bd0a24d5 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:23.030604 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2472141-c171-4d6e-8776-8629bd0a24d5 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:23.084991 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7dd3f4-7d2b-41e6-a63d-21c117f4a442 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:23.107560 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3043b0-ea61-41d4-9552-6aa4b44f8640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:23.109031 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3043b0-ea61-41d4-9552-6aa4b44f8640 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:23.110215 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3043b0-ea61-41d4-9552-6aa4b44f8640 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:23.110990 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3043b0-ea61-41d4-9552-6aa4b44f8640 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:23.131252 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3043b0-ea61-41d4-9552-6aa4b44f8640 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:23.135398 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3043b0-ea61-41d4-9552-6aa4b44f8640 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:23.150627 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:23.170081 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-cb3043b0-ea61-41d4-9552-6aa4b44f8640 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 06 20:00:23.170081 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:23.170081 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:00:23.170081 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:00:23.170081 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:00:23.170081 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:00:23.170081 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:00:23.170081 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:00:23.170081 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:00:23.170081 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:00:23.170081 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:00:23.170081 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:00:23.170081 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 20:00:23.170081 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 20:00:23.170081 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 20:00:23.170081 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 20:00:23.172034 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:00:23.172034 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:00:23.172034 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 20:00:23.172034 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 20:00:23.172034 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 20:00:23.172034 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 20:00:23.172034 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:00:23.172034 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:00:23.172034 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:00:23.172034 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:00:23.172034 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:00:23.172034 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 20:00:23.172034 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:00:23.172034 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:00:23.172034 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:00:23.172034 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:00:23.172034 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:00:23.172034 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:00:23.174058 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:00:23.174058 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:00:23.174058 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 20:00:23.174058 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 20:00:23.174058 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 20:00:23.174058 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 20:00:23.174058 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 20:00:23.174058 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 20:00:23.174058 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 06 20:00:23.174058 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:00:23.176601 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 432/890] 10.222.0.20 () {56 vars in 1117 bytes} [Tue Sep 6 20:00:22 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 357 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:00:23.187159 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3043b0-ea61-41d4-9552-6aa4b44f8640 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:23.217061 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:23.316903 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7dd3f4-7d2b-41e6-a63d-21c117f4a442 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:23.376989 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3043b0-ea61-41d4-9552-6aa4b44f8640 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:23.388621 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a832347-73f1-4d2c-a7e7-14f51bf19256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:23.390664 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a832347-73f1-4d2c-a7e7-14f51bf19256 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:23.391195 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a832347-73f1-4d2c-a7e7-14f51bf19256 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:23.391658 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a832347-73f1-4d2c-a7e7-14f51bf19256 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:23.392462 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a832347-73f1-4d2c-a7e7-14f51bf19256 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:23.395064 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a832347-73f1-4d2c-a7e7-14f51bf19256 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:23.408653 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-6a832347-73f1-4d2c-a7e7-14f51bf19256 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 06 20:00:23.408653 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:23.408653 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 20:00:23.408653 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 20:00:23.408653 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 20:00:23.408653 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 20:00:23.408653 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 20:00:23.408653 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 20:00:23.408653 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 06 20:00:23.408653 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 20:00:23.408653 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 20:00:23.408653 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 20:00:23.408653 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:23.408653 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:00:23.408653 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:00:23.408653 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:00:23.408653 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:00:23.409915 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:00:23.409915 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:00:23.409915 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:00:23.409915 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:00:23.409915 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:00:23.409915 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:00:23.409915 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 20:00:23.409915 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 20:00:23.409915 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 20:00:23.409915 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 20:00:23.409915 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:00:23.409915 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:00:23.409915 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:00:23.409915 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:00:23.409915 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:00:23.409915 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:00:23.410830 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:00:23.410830 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 20:00:23.410830 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:00:23.410830 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:00:23.410830 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:00:23.410830 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:00:23.410830 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:00:23.410830 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:00:23.410830 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:00:23.410830 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:00:23.410830 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 20:00:23.410830 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 20:00:23.410830 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 06 20:00:23.410830 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 20:00:23.414370 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 459/891] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Sep 6 20:00:23 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 20:00:23.423935 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a832347-73f1-4d2c-a7e7-14f51bf19256 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:23.442474 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58d9d06-9750-4da7-b772-3c9c58b93dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:23.443426 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58d9d06-9750-4da7-b772-3c9c58b93dc7 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:23.444012 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58d9d06-9750-4da7-b772-3c9c58b93dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:23.444450 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58d9d06-9750-4da7-b772-3c9c58b93dc7 None admin] PATH_INFO: `/v3/users` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:23.445389 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58d9d06-9750-4da7-b772-3c9c58b93dc7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:23.445886 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58d9d06-9750-4da7-b772-3c9c58b93dc7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:23.448187 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58d9d06-9750-4da7-b772-3c9c58b93dc7 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:23.473793 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 433/892] 10.222.0.20 () {56 vars in 1149 bytes} [Tue Sep 6 20:00:23 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:00:23.483457 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58d9d06-9750-4da7-b772-3c9c58b93dc7 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:23.587077 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:23.632654 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58d9d06-9750-4da7-b772-3c9c58b93dc7 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:23.669866 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:23.677342 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a832347-73f1-4d2c-a7e7-14f51bf19256 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:23.687532 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf9c4e6-846a-4c9d-94ac-5fd6453e2a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:23.688188 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf9c4e6-846a-4c9d-94ac-5fd6453e2a46 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:23.688468 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf9c4e6-846a-4c9d-94ac-5fd6453e2a46 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:23.688736 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf9c4e6-846a-4c9d-94ac-5fd6453e2a46 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:23.755463 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513bf872-a78c-4445-a748-76b53f9a65f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:23.758532 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513bf872-a78c-4445-a748-76b53f9a65f9 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:23.759433 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513bf872-a78c-4445-a748-76b53f9a65f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:23.760539 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513bf872-a78c-4445-a748-76b53f9a65f9 None admin] PATH_INFO: `/v3/services` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:23.763333 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513bf872-a78c-4445-a748-76b53f9a65f9 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:23.764827 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513bf872-a78c-4445-a748-76b53f9a65f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:23.769298 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513bf872-a78c-4445-a748-76b53f9a65f9 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:23.784007 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 460/893] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Sep 6 20:00:23 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 20:00:23.789098 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513bf872-a78c-4445-a748-76b53f9a65f9 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:23.791206 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf9c4e6-846a-4c9d-94ac-5fd6453e2a46 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:23.792922 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf9c4e6-846a-4c9d-94ac-5fd6453e2a46 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:23.954572 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:23.988211 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513bf872-a78c-4445-a748-76b53f9a65f9 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:24.038912 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 434/894] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 6 20:00:23 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 557 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 20:00:24.047888 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf9c4e6-846a-4c9d-94ac-5fd6453e2a46 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:24.065517 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e48cd1-2bef-4950-b1cb-a2c76a4ab411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:24.066258 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e48cd1-2bef-4950-b1cb-a2c76a4ab411 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:24.066777 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e48cd1-2bef-4950-b1cb-a2c76a4ab411 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:24.067242 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e48cd1-2bef-4950-b1cb-a2c76a4ab411 None admin] PATH_INFO: `/v3/services` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:24.068387 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e48cd1-2bef-4950-b1cb-a2c76a4ab411 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:24.069507 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e48cd1-2bef-4950-b1cb-a2c76a4ab411 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:24.071764 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e48cd1-2bef-4950-b1cb-a2c76a4ab411 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:24.083206 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 461/895] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Sep 6 20:00:23 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 20:00:24.095623 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e48cd1-2bef-4950-b1cb-a2c76a4ab411 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:24.167294 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:24.286740 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf9c4e6-846a-4c9d-94ac-5fd6453e2a46 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:24.309240 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e48cd1-2bef-4950-b1cb-a2c76a4ab411 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:24.341976 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c90c54-ab68-4f2e-b825-948d23c1f6ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:24.343072 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c90c54-ab68-4f2e-b825-948d23c1f6ac None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:24.343359 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c90c54-ab68-4f2e-b825-948d23c1f6ac None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:24.343632 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c90c54-ab68-4f2e-b825-948d23c1f6ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:24.358847 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c90c54-ab68-4f2e-b825-948d23c1f6ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:24.360373 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c90c54-ab68-4f2e-b825-948d23c1f6ac None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:24.376005 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a29fb5-2ed1-4ff5-ac26-fe32a850eb86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:24.376831 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a29fb5-2ed1-4ff5-ac26-fe32a850eb86 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:24.377388 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a29fb5-2ed1-4ff5-ac26-fe32a850eb86 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:24.378051 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a29fb5-2ed1-4ff5-ac26-fe32a850eb86 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:24.378623 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-67c90c54-ab68-4f2e-b825-948d23c1f6ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 20:00:24.378623 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:24.378623 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:00:24.378623 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:00:24.378623 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:00:24.378623 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:00:24.378623 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:00:24.378623 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:00:24.378623 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:00:24.378623 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:00:24.378623 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:00:24.378623 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:00:24.378623 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 20:00:24.378623 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 20:00:24.378623 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 20:00:24.378623 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 20:00:24.380373 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:00:24.380373 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:00:24.380373 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:00:24.380373 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:00:24.380373 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:00:24.380373 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:00:24.380373 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:00:24.380373 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 20:00:24.380373 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:00:24.380373 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:00:24.380373 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:00:24.380373 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:00:24.380373 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:00:24.380373 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:00:24.380373 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:00:24.380373 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:00:24.380373 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 20:00:24.383342 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 20:00:24.383342 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 20:00:24.383342 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:00:24.383975 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a29fb5-2ed1-4ff5-ac26-fe32a850eb86 None admin] RBAC: Adding query filter params (interface=public, service_id=2668357f6c0d463cb15ae3b1a206219f, region_id=RegionOne) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:24.383975 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a29fb5-2ed1-4ff5-ac26-fe32a850eb86 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:24.384449 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 435/896] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 20:00:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:00:24.384674 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a29fb5-2ed1-4ff5-ac26-fe32a850eb86 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:24.393706 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 462/897] 10.222.0.20 () {56 vars in 1251 bytes} [Tue Sep 6 20:00:24 2022] GET /identity/v3/endpoints?service_id=2668357f6c0d463cb15ae3b1a206219f&interface=public®ion_id=RegionOne => generated 194 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 20:00:24.399199 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c90c54-ab68-4f2e-b825-948d23c1f6ac None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:24.401552 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a29fb5-2ed1-4ff5-ac26-fe32a850eb86 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:24.450678 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:24.564735 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a29fb5-2ed1-4ff5-ac26-fe32a850eb86 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:24.577850 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:24.614539 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11c1a93-36b7-4880-ae68-c1a1b8edb4c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:24.615332 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11c1a93-36b7-4880-ae68-c1a1b8edb4c8 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:24.615711 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11c1a93-36b7-4880-ae68-c1a1b8edb4c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:24.616009 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11c1a93-36b7-4880-ae68-c1a1b8edb4c8 None admin] PATH_INFO: `/v3/services` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:24.616384 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c90c54-ab68-4f2e-b825-948d23c1f6ac None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:24.616820 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11c1a93-36b7-4880-ae68-c1a1b8edb4c8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:24.618350 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11c1a93-36b7-4880-ae68-c1a1b8edb4c8 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:24.632326 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 463/898] 10.222.0.20 () {56 vars in 1087 bytes} [Tue Sep 6 20:00:24 2022] GET /identity/v3/services => generated 1268 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:00:24.677094 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e889d05-4424-483b-8cec-359ea1330d5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:24.677954 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e889d05-4424-483b-8cec-359ea1330d5c None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:24.678291 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e889d05-4424-483b-8cec-359ea1330d5c None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:24.678552 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e889d05-4424-483b-8cec-359ea1330d5c None admin] PATH_INFO: `/v3/domains` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:24.679184 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e889d05-4424-483b-8cec-359ea1330d5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:24.679618 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e889d05-4424-483b-8cec-359ea1330d5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:24.681268 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e889d05-4424-483b-8cec-359ea1330d5c None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:24.698746 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 436/899] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 6 20:00:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 20:00:24.705507 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11c1a93-36b7-4880-ae68-c1a1b8edb4c8 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:24.886414 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11c1a93-36b7-4880-ae68-c1a1b8edb4c8 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:24.958371 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f57ba0-0383-4ed2-a932-e751575cf758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:24.959560 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f57ba0-0383-4ed2-a932-e751575cf758 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:24.960122 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f57ba0-0383-4ed2-a932-e751575cf758 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:24.960617 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f57ba0-0383-4ed2-a932-e751575cf758 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:24.974424 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f57ba0-0383-4ed2-a932-e751575cf758 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:24.977234 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f57ba0-0383-4ed2-a932-e751575cf758 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:25.001214 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-e6f57ba0-0383-4ed2-a932-e751575cf758 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 20:00:25.001214 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:25.001214 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:00:25.001214 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:00:25.001214 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:00:25.001214 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:00:25.001214 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:00:25.001214 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:00:25.001214 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:00:25.001214 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:00:25.001214 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:00:25.001214 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:00:25.001214 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 20:00:25.001214 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 20:00:25.001214 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 20:00:25.002974 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 20:00:25.002974 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:00:25.002974 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:00:25.002974 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:00:25.002974 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:00:25.002974 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:00:25.002974 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:00:25.002974 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:00:25.002974 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 20:00:25.002974 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:00:25.002974 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:00:25.002974 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:00:25.002974 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:00:25.002974 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:00:25.002974 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:00:25.002974 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:00:25.002974 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:00:25.005907 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 20:00:25.005907 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 20:00:25.005907 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 20:00:25.005907 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 20:00:25.005907 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 20:00:25.005907 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 20:00:25.005907 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 20:00:25.005907 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 20:00:25.010250 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 464/900] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 6 20:00:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:00:25.014524 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e889d05-4424-483b-8cec-359ea1330d5c None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:25.218716 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:25.242700 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e889d05-4424-483b-8cec-359ea1330d5c None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:25.307702 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9606cdce-9277-440e-835c-26d7494b9bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:25.309034 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9606cdce-9277-440e-835c-26d7494b9bcb None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:25.309470 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9606cdce-9277-440e-835c-26d7494b9bcb None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:25.309847 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9606cdce-9277-440e-835c-26d7494b9bcb None admin] PATH_INFO: `/v3/projects` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:25.311005 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9606cdce-9277-440e-835c-26d7494b9bcb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:25.311497 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9606cdce-9277-440e-835c-26d7494b9bcb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:25.313737 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9606cdce-9277-440e-835c-26d7494b9bcb None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:25.339731 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 437/901] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Sep 6 20:00:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:00:25.347992 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f57ba0-0383-4ed2-a932-e751575cf758 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:25.376761 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:25.502152 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f57ba0-0383-4ed2-a932-e751575cf758 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:25.554100 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f83e50c-07f9-49fc-9b3c-2de278a93eda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:25.555204 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f83e50c-07f9-49fc-9b3c-2de278a93eda None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:25.555782 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f83e50c-07f9-49fc-9b3c-2de278a93eda None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:25.556370 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f83e50c-07f9-49fc-9b3c-2de278a93eda None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:25.557566 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f83e50c-07f9-49fc-9b3c-2de278a93eda None admin] RBAC: Adding query filter params (role.id=960d5e4b29394e7aae95d5a428dedfc0, scope.project.id=6aef43493bb1420a9e122313d8db7d5a, user.id=1a52117df70447dba9bbee35d083e44f) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:25.558294 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f83e50c-07f9-49fc-9b3c-2de278a93eda None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:25.561145 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f83e50c-07f9-49fc-9b3c-2de278a93eda None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:25.573790 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 465/902] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Sep 6 20:00:25 2022] GET /identity/v3/role_assignments?user.id=1a52117df70447dba9bbee35d083e44f&scope.project.id=6aef43493bb1420a9e122313d8db7d5a&role.id=960d5e4b29394e7aae95d5a428dedfc0 => generated 259 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:00:25.618141 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ef5fe7-ea64-48d5-bd09-a948780f6f2e None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:25.618753 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ef5fe7-ea64-48d5-bd09-a948780f6f2e None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:25.619170 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ef5fe7-ea64-48d5-bd09-a948780f6f2e None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:25.623250 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 438/903] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:00:25 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:00:25.629279 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa714c63-a272-48cb-ab01-800c9f603b83 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:25.629575 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa714c63-a272-48cb-ab01-800c9f603b83 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:25.630105 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa714c63-a272-48cb-ab01-800c9f603b83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:25.653678 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-aa714c63-a272-48cb-ab01-800c9f603b83 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:00:25.654707 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-aa714c63-a272-48cb-ab01-800c9f603b83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:00:25.754951 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 466/904] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:00:25 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 20:00:26.061785 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ef5fe7-ea64-48d5-bd09-a948780f6f2e None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:26.203073 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ef5fe7-ea64-48d5-bd09-a948780f6f2e None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:26.244398 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2d04f8-9cb9-4a04-b963-4451599ecb56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:26.245097 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2d04f8-9cb9-4a04-b963-4451599ecb56 None admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:26.245433 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2d04f8-9cb9-4a04-b963-4451599ecb56 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:26.245748 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2d04f8-9cb9-4a04-b963-4451599ecb56 None admin] PATH_INFO: `/v3/services` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:26.246506 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2d04f8-9cb9-4a04-b963-4451599ecb56 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:26.248700 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2d04f8-9cb9-4a04-b963-4451599ecb56 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:26.268445 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 439/905] 10.222.0.20 () {60 vars in 1141 bytes} [Tue Sep 6 20:00:26 2022] POST /identity/v3/services => generated 241 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:00:26.296301 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8046258-2948-493b-9c02-9eac5f914b22 None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:26.297754 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8046258-2948-493b-9c02-9eac5f914b22 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:26.297754 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8046258-2948-493b-9c02-9eac5f914b22 None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:26.304448 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 467/906] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:00:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:00:26.308057 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616fe876-aa14-463d-a4e5-a24c343f3541 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:26.308451 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616fe876-aa14-463d-a4e5-a24c343f3541 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:26.308756 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616fe876-aa14-463d-a4e5-a24c343f3541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:26.318132 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:26.333157 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-616fe876-aa14-463d-a4e5-a24c343f3541 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:00:26.334655 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-616fe876-aa14-463d-a4e5-a24c343f3541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:00:26.437775 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 440/907] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:00:26 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 20:00:26.726830 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8046258-2948-493b-9c02-9eac5f914b22 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:26.743617 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:26.917788 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8046258-2948-493b-9c02-9eac5f914b22 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:26.982688 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13735571-114b-4892-81e7-61221ea36a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:26.983762 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13735571-114b-4892-81e7-61221ea36a56 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:26.984249 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13735571-114b-4892-81e7-61221ea36a56 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:26.984720 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13735571-114b-4892-81e7-61221ea36a56 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:26.985800 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13735571-114b-4892-81e7-61221ea36a56 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:26.988315 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13735571-114b-4892-81e7-61221ea36a56 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:27.006022 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-13735571-114b-4892-81e7-61221ea36a56 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 06 20:00:27.006022 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:27.006022 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 20:00:27.006022 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 20:00:27.006022 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 20:00:27.006022 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 20:00:27.006022 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 20:00:27.006022 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 20:00:27.006022 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 06 20:00:27.006022 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 20:00:27.006022 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 20:00:27.006022 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 20:00:27.006022 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:27.006022 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:00:27.006022 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:00:27.006022 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:00:27.007646 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:00:27.007646 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:00:27.007646 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:00:27.007646 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:00:27.007646 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:00:27.007646 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:00:27.007646 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:00:27.007646 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 20:00:27.007646 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 20:00:27.007646 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 20:00:27.007646 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 20:00:27.007646 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:00:27.007646 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:00:27.007646 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:00:27.007646 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:00:27.007646 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:00:27.007646 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:00:27.009067 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:00:27.009067 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 20:00:27.009067 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:00:27.009067 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:00:27.009067 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:00:27.009067 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:00:27.009067 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:00:27.009067 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:00:27.009067 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:00:27.009067 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:00:27.009067 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 20:00:27.009067 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 20:00:27.009067 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 06 20:00:27.009067 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 20:00:27.009067 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 468/908] 10.222.0.20 () {56 vars in 1132 bytes} [Tue Sep 6 20:00:26 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:00:27.020195 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616fe876-aa14-463d-a4e5-a24c343f3541 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:27.233025 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616fe876-aa14-463d-a4e5-a24c343f3541 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:27.302400 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:27.312961 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3520b76d-6d6e-4b34-b2d9-5c87fb7190a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:27.314049 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3520b76d-6d6e-4b34-b2d9-5c87fb7190a5 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:27.314537 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3520b76d-6d6e-4b34-b2d9-5c87fb7190a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:27.315015 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3520b76d-6d6e-4b34-b2d9-5c87fb7190a5 None admin] PATH_INFO: `/v3/services` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:27.316257 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3520b76d-6d6e-4b34-b2d9-5c87fb7190a5 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:27.316782 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3520b76d-6d6e-4b34-b2d9-5c87fb7190a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:27.319490 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3520b76d-6d6e-4b34-b2d9-5c87fb7190a5 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:27.330919 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 441/909] 10.222.0.20 () {56 vars in 1126 bytes} [Tue Sep 6 20:00:27 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:00:27.341042 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13735571-114b-4892-81e7-61221ea36a56 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:27.506299 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:27.561679 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13735571-114b-4892-81e7-61221ea36a56 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:27.640504 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c3fcb6-1ffb-40c4-a651-6fc2dffcdd1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:27.644961 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c3fcb6-1ffb-40c4-a651-6fc2dffcdd1a None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:27.644961 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c3fcb6-1ffb-40c4-a651-6fc2dffcdd1a None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:27.644961 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c3fcb6-1ffb-40c4-a651-6fc2dffcdd1a None admin] PATH_INFO: `/v3/services` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:27.644961 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c3fcb6-1ffb-40c4-a651-6fc2dffcdd1a None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:27.644961 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c3fcb6-1ffb-40c4-a651-6fc2dffcdd1a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:27.647198 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c3fcb6-1ffb-40c4-a651-6fc2dffcdd1a None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:27.662777 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 469/910] 10.222.0.20 () {56 vars in 1126 bytes} [Tue Sep 6 20:00:27 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:00:27.672352 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3520b76d-6d6e-4b34-b2d9-5c87fb7190a5 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:27.713217 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb92da24-80e1-4496-a917-b381c2497cf2 None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:27.714043 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb92da24-80e1-4496-a917-b381c2497cf2 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:27.714663 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb92da24-80e1-4496-a917-b381c2497cf2 None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:27.718381 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 470/911] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:00:27 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:00:27.723908 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db17a1f-55da-47f9-8ddd-2c16bac247f7 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:27.724205 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db17a1f-55da-47f9-8ddd-2c16bac247f7 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:27.724506 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db17a1f-55da-47f9-8ddd-2c16bac247f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:27.745987 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-8db17a1f-55da-47f9-8ddd-2c16bac247f7 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:00:27.748148 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-8db17a1f-55da-47f9-8ddd-2c16bac247f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:00:27.882573 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 471/912] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:00:27 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 20:00:27.885837 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3520b76d-6d6e-4b34-b2d9-5c87fb7190a5 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:27.957443 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaa8c50-cbf4-44a6-937a-413f3886a26d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:27.958920 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaa8c50-cbf4-44a6-937a-413f3886a26d None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:27.959605 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaa8c50-cbf4-44a6-937a-413f3886a26d None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:27.960279 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaa8c50-cbf4-44a6-937a-413f3886a26d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:27.961900 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaa8c50-cbf4-44a6-937a-413f3886a26d None admin] RBAC: Adding query filter params (interface=public, service_id=924c5685121b4c04a804d6084cd212df, region_id=RegionOne) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:27.962652 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaa8c50-cbf4-44a6-937a-413f3886a26d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:27.965838 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaa8c50-cbf4-44a6-937a-413f3886a26d None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:27.982446 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 442/913] 10.222.0.20 () {56 vars in 1251 bytes} [Tue Sep 6 20:00:27 2022] GET /identity/v3/endpoints?service_id=924c5685121b4c04a804d6084cd212df&interface=public®ion_id=RegionOne => generated 194 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:00:27.992438 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db17a1f-55da-47f9-8ddd-2c16bac247f7 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:28.131562 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:28.176445 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaa8c50-cbf4-44a6-937a-413f3886a26d None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:28.208006 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db17a1f-55da-47f9-8ddd-2c16bac247f7 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:28.293266 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f28248-a065-43ef-9acb-887491eb3f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:28.296002 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f28248-a065-43ef-9acb-887491eb3f8a None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:28.296881 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f28248-a065-43ef-9acb-887491eb3f8a None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:28.297749 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f28248-a065-43ef-9acb-887491eb3f8a None admin] PATH_INFO: `/v3/services` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:28.299246 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f28248-a065-43ef-9acb-887491eb3f8a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:28.303457 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f28248-a065-43ef-9acb-887491eb3f8a None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:28.313605 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:28.326251 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 472/914] 10.222.0.20 () {56 vars in 1087 bytes} [Tue Sep 6 20:00:27 2022] GET /identity/v3/services => generated 1497 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:00:28.414706 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaa8c50-cbf4-44a6-937a-413f3886a26d None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:28.489862 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938d1f2b-ba74-427c-b276-90c64cc7e490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:28.490911 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938d1f2b-ba74-427c-b276-90c64cc7e490 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:28.491434 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938d1f2b-ba74-427c-b276-90c64cc7e490 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:28.491923 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938d1f2b-ba74-427c-b276-90c64cc7e490 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:28.493053 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938d1f2b-ba74-427c-b276-90c64cc7e490 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:28.496027 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938d1f2b-ba74-427c-b276-90c64cc7e490 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:28.516289 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-938d1f2b-ba74-427c-b276-90c64cc7e490 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 06 20:00:28.516289 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:28.516289 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 20:00:28.516289 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 20:00:28.516289 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 20:00:28.516289 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 20:00:28.516289 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 20:00:28.516289 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 20:00:28.516289 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 06 20:00:28.516289 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:00:28.516289 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 20:00:28.516289 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:00:28.516289 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:28.516289 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:00:28.516289 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:00:28.516289 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:00:28.516289 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:00:28.518126 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:00:28.518126 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:00:28.518126 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:00:28.518126 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:00:28.518126 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:00:28.518126 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:00:28.518126 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 20:00:28.518126 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 20:00:28.518126 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 20:00:28.518126 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 20:00:28.518126 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:00:28.518126 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:00:28.518126 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:00:28.518126 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:00:28.518126 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:00:28.518126 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:00:28.520200 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:00:28.520200 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 20:00:28.520200 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:00:28.520200 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:00:28.520200 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:00:28.520200 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:00:28.520200 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:00:28.520200 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:00:28.520200 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:00:28.520200 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:00:28.520200 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 20:00:28.520200 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 20:00:28.520200 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 06 20:00:28.520200 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:00:28.520200 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 443/915] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Sep 6 20:00:28 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:00:28.530894 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f28248-a065-43ef-9acb-887491eb3f8a None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:28.657992 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ab7231-1771-4793-8538-9a9f106ccb1f None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:28.658336 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ab7231-1771-4793-8538-9a9f106ccb1f None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:28.658669 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ab7231-1771-4793-8538-9a9f106ccb1f None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:28.660201 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 444/916] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:00:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 20:00:28.667076 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b481a2-7bf9-42b9-b883-9dac6eb24c40 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:28.667542 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b481a2-7bf9-42b9-b883-9dac6eb24c40 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:28.668103 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b481a2-7bf9-42b9-b883-9dac6eb24c40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:28.691535 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f28248-a065-43ef-9acb-887491eb3f8a None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:28.704480 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-45b481a2-7bf9-42b9-b883-9dac6eb24c40 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:00:28.706664 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-45b481a2-7bf9-42b9-b883-9dac6eb24c40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:00:28.773499 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:28.787764 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3b7f30-4ab3-409c-8ac7-777cca53a148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:28.790088 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3b7f30-4ab3-409c-8ac7-777cca53a148 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:28.791461 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3b7f30-4ab3-409c-8ac7-777cca53a148 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:28.793025 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3b7f30-4ab3-409c-8ac7-777cca53a148 None admin] PATH_INFO: `/v3/services` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:28.794634 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3b7f30-4ab3-409c-8ac7-777cca53a148 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:28.794952 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3b7f30-4ab3-409c-8ac7-777cca53a148 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:28.796500 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3b7f30-4ab3-409c-8ac7-777cca53a148 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:28.809542 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 473/917] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Sep 6 20:00:28 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:00:28.811130 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 445/918] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:00:28 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 20:00:28.815817 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3b7f30-4ab3-409c-8ac7-777cca53a148 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:29.032041 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3b7f30-4ab3-409c-8ac7-777cca53a148 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:29.041997 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff636ab-2efa-45a3-92f7-21bd69d98f46 None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:29.042285 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff636ab-2efa-45a3-92f7-21bd69d98f46 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:29.042623 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff636ab-2efa-45a3-92f7-21bd69d98f46 None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:29.045002 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 446/919] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:00:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:00:29.051121 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aba533-f82f-4ec4-9c8c-c754b7f75201 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:29.051767 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aba533-f82f-4ec4-9c8c-c754b7f75201 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:29.051767 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aba533-f82f-4ec4-9c8c-c754b7f75201 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:29.072798 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-66aba533-f82f-4ec4-9c8c-c754b7f75201 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:00:29.074087 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-66aba533-f82f-4ec4-9c8c-c754b7f75201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:00:29.110603 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e17b10a-240b-4e4c-9f72-f1b041012422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:29.111645 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e17b10a-240b-4e4c-9f72-f1b041012422 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:29.112128 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e17b10a-240b-4e4c-9f72-f1b041012422 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:29.112633 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e17b10a-240b-4e4c-9f72-f1b041012422 None admin] PATH_INFO: `/v3/services` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:29.113894 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e17b10a-240b-4e4c-9f72-f1b041012422 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:29.114439 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e17b10a-240b-4e4c-9f72-f1b041012422 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:29.117141 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e17b10a-240b-4e4c-9f72-f1b041012422 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:29.129849 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 474/920] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Sep 6 20:00:28 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:00:29.129849 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e17b10a-240b-4e4c-9f72-f1b041012422 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:29.165799 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:29.193934 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 447/921] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:00:29 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 20:00:29.193934 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aba533-f82f-4ec4-9c8c-c754b7f75201 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:29.223957 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:29.304154 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e17b10a-240b-4e4c-9f72-f1b041012422 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:29.331972 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aba533-f82f-4ec4-9c8c-c754b7f75201 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:29.372704 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c48afd-57d7-4c9d-811b-b3c197248763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:29.373944 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c48afd-57d7-4c9d-811b-b3c197248763 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:29.374695 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c48afd-57d7-4c9d-811b-b3c197248763 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:29.375015 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c48afd-57d7-4c9d-811b-b3c197248763 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:29.380518 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2a800b-23a9-425e-a1ab-73f9499ea217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:29.381488 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2a800b-23a9-425e-a1ab-73f9499ea217 None admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:29.381960 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2a800b-23a9-425e-a1ab-73f9499ea217 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:29.382398 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2a800b-23a9-425e-a1ab-73f9499ea217 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:29.383237 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2a800b-23a9-425e-a1ab-73f9499ea217 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:29.384782 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2a800b-23a9-425e-a1ab-73f9499ea217 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:29.396177 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c48afd-57d7-4c9d-811b-b3c197248763 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 20:00:29.408266 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-85c48afd-57d7-4c9d-811b-b3c197248763 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 06 20:00:29.408266 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:29.408266 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:00:29.408266 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:00:29.408266 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:00:29.408266 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:00:29.408266 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:00:29.408266 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:00:29.408266 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:00:29.408266 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:00:29.408266 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:00:29.408266 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:00:29.408266 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 20:00:29.408266 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 20:00:29.408266 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 20:00:29.408266 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 20:00:29.409676 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 20:00:29.409676 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 20:00:29.409676 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 20:00:29.409676 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 20:00:29.409676 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:00:29.409676 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:00:29.409676 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:00:29.409676 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:00:29.409676 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:00:29.409676 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:00:29.409676 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:00:29.409676 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 20:00:29.409676 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:00:29.409676 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:00:29.409676 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:00:29.409676 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:00:29.410899 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:00:29.410899 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:00:29.410899 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:00:29.410899 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:00:29.410899 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 20:00:29.410899 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 20:00:29.410899 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 20:00:29.410899 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 20:00:29.410899 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 20:00:29.410899 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 20:00:29.410899 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 06 20:00:29.410899 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 20:00:29.418407 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 475/922] 10.222.0.20 () {56 vars in 1120 bytes} [Tue Sep 6 20:00:29 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:00:29.420399 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 448/923] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:00:29 2022] POST /identity/v3/endpoints => generated 346 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:00:29.425763 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c48afd-57d7-4c9d-811b-b3c197248763 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:29.630708 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c48afd-57d7-4c9d-811b-b3c197248763 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:29.671670 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:29.688085 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2a800b-23a9-425e-a1ab-73f9499ea217 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:29.709191 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6182520-cc53-487d-b447-84d339fba02d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:29.710119 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6182520-cc53-487d-b447-84d339fba02d None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:29.710532 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6182520-cc53-487d-b447-84d339fba02d None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:29.710945 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6182520-cc53-487d-b447-84d339fba02d None admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:29.711806 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6182520-cc53-487d-b447-84d339fba02d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:29.712332 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6182520-cc53-487d-b447-84d339fba02d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:29.714113 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6182520-cc53-487d-b447-84d339fba02d None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:29.728405 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 476/924] 10.222.0.20 () {56 vars in 1115 bytes} [Tue Sep 6 20:00:29 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 20:00:29.736230 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6182520-cc53-487d-b447-84d339fba02d None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:29.753171 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:29.845137 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2a800b-23a9-425e-a1ab-73f9499ea217 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:29.890230 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1c4fad-8fcc-4121-a102-2cfe5be02799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:29.891402 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1c4fad-8fcc-4121-a102-2cfe5be02799 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:29.891950 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1c4fad-8fcc-4121-a102-2cfe5be02799 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:29.892439 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1c4fad-8fcc-4121-a102-2cfe5be02799 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:29.893344 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1c4fad-8fcc-4121-a102-2cfe5be02799 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:29.895072 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1c4fad-8fcc-4121-a102-2cfe5be02799 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:29.907362 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-db1c4fad-8fcc-4121-a102-2cfe5be02799 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 20:00:29.907362 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:29.907362 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 20:00:29.907362 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 20:00:29.907362 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 20:00:29.907362 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 20:00:29.907362 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 20:00:29.907362 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 20:00:29.907362 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 20:00:29.907362 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:00:29.907362 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 20:00:29.907362 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:00:29.907362 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:29.907362 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:00:29.907362 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:00:29.907362 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:00:29.907362 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:00:29.909293 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:00:29.909293 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:00:29.909293 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:00:29.909293 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:00:29.909293 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:00:29.909293 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:00:29.909293 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 20:00:29.909293 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 20:00:29.909293 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 20:00:29.909293 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 20:00:29.909293 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:00:29.909293 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:00:29.909293 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:00:29.909293 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:00:29.909293 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:00:29.909293 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:00:29.911310 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:00:29.911310 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 20:00:29.911310 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:00:29.911310 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:00:29.911310 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:00:29.911310 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:00:29.911310 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:00:29.911310 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:00:29.911310 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:00:29.911310 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:00:29.911310 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 20:00:29.911310 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 20:00:29.911310 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 20:00:29.911310 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:00:29.917721 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 449/925] 10.222.0.20 () {56 vars in 1126 bytes} [Tue Sep 6 20:00:29 2022] GET /identity/v3/services/object-store => generated 93 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:00:29.921321 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1c4fad-8fcc-4121-a102-2cfe5be02799 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:29.931654 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6182520-cc53-487d-b447-84d339fba02d None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:30.001170 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a3dce0-c911-43c3-9787-827e56311554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:30.002277 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a3dce0-c911-43c3-9787-827e56311554 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:30.002716 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a3dce0-c911-43c3-9787-827e56311554 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:30.003194 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a3dce0-c911-43c3-9787-827e56311554 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:30.023674 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a3dce0-c911-43c3-9787-827e56311554 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:30.026539 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a3dce0-c911-43c3-9787-827e56311554 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:30.051887 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1c4fad-8fcc-4121-a102-2cfe5be02799 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:30.054190 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:30.055843 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-40a3dce0-c911-43c3-9787-827e56311554 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 20:00:30.055843 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:30.055843 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:00:30.055843 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:00:30.055843 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:00:30.055843 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:00:30.055843 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:00:30.055843 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:00:30.055843 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:00:30.055843 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:00:30.055843 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:00:30.055843 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:00:30.055843 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 20:00:30.055843 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 20:00:30.055843 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 20:00:30.055843 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 20:00:30.057213 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:00:30.057213 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:00:30.057213 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:00:30.057213 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:00:30.057213 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:00:30.057213 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:00:30.057213 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:00:30.057213 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 20:00:30.057213 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:00:30.057213 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:00:30.057213 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:00:30.057213 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:00:30.057213 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:00:30.057213 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:00:30.057213 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:00:30.057213 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:00:30.057213 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 20:00:30.058204 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 20:00:30.058204 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 20:00:30.058204 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 20:00:30.062193 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 477/926] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 20:00:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 20:00:30.068456 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a3dce0-c911-43c3-9787-827e56311554 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:30.099295 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a37570-6e0e-4668-a8da-a73d6b61eae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:30.100301 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a37570-6e0e-4668-a8da-a73d6b61eae7 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:30.100757 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a37570-6e0e-4668-a8da-a73d6b61eae7 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:30.101293 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a37570-6e0e-4668-a8da-a73d6b61eae7 None admin] PATH_INFO: `/v3/services` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:30.102179 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a37570-6e0e-4668-a8da-a73d6b61eae7 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:30.102668 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a37570-6e0e-4668-a8da-a73d6b61eae7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:30.104485 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a37570-6e0e-4668-a8da-a73d6b61eae7 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:30.116075 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 450/927] 10.222.0.20 () {56 vars in 1122 bytes} [Tue Sep 6 20:00:29 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 20:00:30.118413 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a37570-6e0e-4668-a8da-a73d6b61eae7 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:30.129245 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:30.249055 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a37570-6e0e-4668-a8da-a73d6b61eae7 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:30.272181 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a3dce0-c911-43c3-9787-827e56311554 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:30.296482 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51df3941-b338-48b5-a5de-f8340705fca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:30.298263 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51df3941-b338-48b5-a5de-f8340705fca7 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:30.299186 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51df3941-b338-48b5-a5de-f8340705fca7 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:30.299981 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51df3941-b338-48b5-a5de-f8340705fca7 None admin] PATH_INFO: `/v3/services` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:30.301510 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51df3941-b338-48b5-a5de-f8340705fca7 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:30.302463 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51df3941-b338-48b5-a5de-f8340705fca7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:30.305749 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51df3941-b338-48b5-a5de-f8340705fca7 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:30.319010 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 451/928] 10.222.0.20 () {56 vars in 1122 bytes} [Tue Sep 6 20:00:30 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 20:00:30.320425 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51df3941-b338-48b5-a5de-f8340705fca7 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:30.337486 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88dc917-3a00-469d-93de-b5a080a56740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:30.338548 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88dc917-3a00-469d-93de-b5a080a56740 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:30.339027 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88dc917-3a00-469d-93de-b5a080a56740 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:30.339567 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88dc917-3a00-469d-93de-b5a080a56740 None admin] PATH_INFO: `/v3/domains` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:30.340824 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88dc917-3a00-469d-93de-b5a080a56740 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:30.341342 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88dc917-3a00-469d-93de-b5a080a56740 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:30.344053 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88dc917-3a00-469d-93de-b5a080a56740 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:30.363420 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:30.368670 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 478/929] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 6 20:00:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:00:30.377826 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88dc917-3a00-469d-93de-b5a080a56740 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:30.460192 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51df3941-b338-48b5-a5de-f8340705fca7 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:30.504358 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04438d97-0692-43ca-b92d-cf704c9f90c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:30.505387 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04438d97-0692-43ca-b92d-cf704c9f90c8 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:30.505922 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04438d97-0692-43ca-b92d-cf704c9f90c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:30.506410 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04438d97-0692-43ca-b92d-cf704c9f90c8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:30.507366 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04438d97-0692-43ca-b92d-cf704c9f90c8 None admin] RBAC: Adding query filter params (interface=public, service_id=51bf00ccd5d84eef914a4ece4b88a52e, region_id=RegionOne) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:30.507986 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04438d97-0692-43ca-b92d-cf704c9f90c8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:30.509767 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04438d97-0692-43ca-b92d-cf704c9f90c8 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:30.522767 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 452/930] 10.222.0.20 () {56 vars in 1251 bytes} [Tue Sep 6 20:00:30 2022] GET /identity/v3/endpoints?service_id=51bf00ccd5d84eef914a4ece4b88a52e&interface=public®ion_id=RegionOne => generated 194 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:00:30.524141 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04438d97-0692-43ca-b92d-cf704c9f90c8 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:30.555717 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:30.574742 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88dc917-3a00-469d-93de-b5a080a56740 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:30.644272 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93442fdc-3c1e-4858-9c5a-a7d4cf09108c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:30.645279 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93442fdc-3c1e-4858-9c5a-a7d4cf09108c None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:30.645836 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93442fdc-3c1e-4858-9c5a-a7d4cf09108c None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:30.646310 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93442fdc-3c1e-4858-9c5a-a7d4cf09108c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:30.661159 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93442fdc-3c1e-4858-9c5a-a7d4cf09108c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:30.664120 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93442fdc-3c1e-4858-9c5a-a7d4cf09108c None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:30.665043 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:30.675413 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04438d97-0692-43ca-b92d-cf704c9f90c8 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:30.688204 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-93442fdc-3c1e-4858-9c5a-a7d4cf09108c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 06 20:00:30.688204 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:30.688204 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:00:30.688204 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:00:30.688204 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:00:30.688204 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:00:30.688204 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:00:30.688204 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:00:30.688204 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:00:30.688204 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:00:30.688204 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:00:30.688204 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:00:30.688204 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 20:00:30.688204 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 20:00:30.688204 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 20:00:30.688204 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 20:00:30.689412 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:00:30.689412 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:00:30.689412 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 20:00:30.689412 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 20:00:30.689412 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 20:00:30.689412 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 20:00:30.689412 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:00:30.689412 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:00:30.689412 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:00:30.689412 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:00:30.689412 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:00:30.689412 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 20:00:30.689412 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:00:30.689412 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:00:30.689412 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:00:30.689412 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:00:30.689412 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:00:30.689412 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:00:30.690425 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:00:30.690425 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:00:30.690425 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 20:00:30.690425 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 20:00:30.690425 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 20:00:30.690425 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 20:00:30.690425 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 20:00:30.690425 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 20:00:30.690425 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 06 20:00:30.690425 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 20:00:30.698066 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 479/931] 10.222.0.20 () {56 vars in 1117 bytes} [Tue Sep 6 20:00:30 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:00:30.699759 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93442fdc-3c1e-4858-9c5a-a7d4cf09108c None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:30.723216 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c92385-5f9d-4103-9983-5b880ec17c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:30.724334 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c92385-5f9d-4103-9983-5b880ec17c65 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:30.724928 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c92385-5f9d-4103-9983-5b880ec17c65 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:30.725435 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c92385-5f9d-4103-9983-5b880ec17c65 None admin] PATH_INFO: `/v3/services` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:30.726358 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c92385-5f9d-4103-9983-5b880ec17c65 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:30.728056 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c92385-5f9d-4103-9983-5b880ec17c65 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:30.735772 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 453/932] 10.222.0.20 () {56 vars in 1087 bytes} [Tue Sep 6 20:00:30 2022] GET /identity/v3/services => generated 1497 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:00:30.911293 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93442fdc-3c1e-4858-9c5a-a7d4cf09108c None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:30.983452 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc73db0-1e03-4fa7-9602-66498f230a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:30.984422 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc73db0-1e03-4fa7-9602-66498f230a8b None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:30.984860 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc73db0-1e03-4fa7-9602-66498f230a8b None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:30.985307 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc73db0-1e03-4fa7-9602-66498f230a8b None admin] PATH_INFO: `/v3/users` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:30.986708 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc73db0-1e03-4fa7-9602-66498f230a8b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:30.987238 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc73db0-1e03-4fa7-9602-66498f230a8b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:30.989944 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc73db0-1e03-4fa7-9602-66498f230a8b None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:31.028346 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 480/933] 10.222.0.20 () {56 vars in 1149 bytes} [Tue Sep 6 20:00:30 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:00:31.037779 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c92385-5f9d-4103-9983-5b880ec17c65 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:31.193710 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c287b671-0905-4c96-bc6e-92d74531a32c None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:31.193995 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c287b671-0905-4c96-bc6e-92d74531a32c None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:31.194254 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c287b671-0905-4c96-bc6e-92d74531a32c None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:31.197555 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 481/934] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:00:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 20:00:31.203121 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e936ed89-44db-4d6b-80d9-a410a57e7077 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:31.203456 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e936ed89-44db-4d6b-80d9-a410a57e7077 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:31.203697 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e936ed89-44db-4d6b-80d9-a410a57e7077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:31.224625 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-e936ed89-44db-4d6b-80d9-a410a57e7077 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:00:31.226036 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-e936ed89-44db-4d6b-80d9-a410a57e7077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:00:31.228389 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:31.254720 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c92385-5f9d-4103-9983-5b880ec17c65 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:31.329855 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 482/935] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:00:31 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 20:00:31.333905 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a869c741-74af-4008-a830-c3a487fcab0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:31.334980 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a869c741-74af-4008-a830-c3a487fcab0e None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:31.335462 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a869c741-74af-4008-a830-c3a487fcab0e None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:31.335936 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a869c741-74af-4008-a830-c3a487fcab0e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:31.356293 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a869c741-74af-4008-a830-c3a487fcab0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:31.358178 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a869c741-74af-4008-a830-c3a487fcab0e None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:31.376788 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-a869c741-74af-4008-a830-c3a487fcab0e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 20:00:31.376788 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:31.376788 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:00:31.376788 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:00:31.376788 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:00:31.376788 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:00:31.376788 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:00:31.376788 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:00:31.376788 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:00:31.376788 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:00:31.376788 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:00:31.376788 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:00:31.376788 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 20:00:31.376788 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 20:00:31.376788 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 20:00:31.376788 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 20:00:31.379208 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:00:31.379208 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:00:31.379208 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:00:31.379208 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:00:31.379208 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:00:31.379208 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:00:31.379208 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:00:31.379208 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 20:00:31.379208 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:00:31.379208 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:00:31.379208 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:00:31.379208 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:00:31.379208 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:00:31.379208 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:00:31.379208 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:00:31.379208 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:00:31.379208 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 20:00:31.380567 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 20:00:31.380567 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 20:00:31.380567 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:00:31.380567 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 454/936] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 20:00:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:00:31.387033 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e936ed89-44db-4d6b-80d9-a410a57e7077 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:31.564481 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e936ed89-44db-4d6b-80d9-a410a57e7077 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:31.626196 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a869c741-74af-4008-a830-c3a487fcab0e None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:31.636673 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f865bcb0-2f06-42af-9fa4-ade56fa60a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:31.637723 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f865bcb0-2f06-42af-9fa4-ade56fa60a21 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:31.638205 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f865bcb0-2f06-42af-9fa4-ade56fa60a21 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:31.638695 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f865bcb0-2f06-42af-9fa4-ade56fa60a21 None admin] PATH_INFO: `/v3/domains` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:31.639664 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f865bcb0-2f06-42af-9fa4-ade56fa60a21 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:31.639984 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f865bcb0-2f06-42af-9fa4-ade56fa60a21 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:31.644721 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f865bcb0-2f06-42af-9fa4-ade56fa60a21 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:31.650711 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:31.657256 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:31.661552 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 483/937] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 6 20:00:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:00:31.667990 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f865bcb0-2f06-42af-9fa4-ade56fa60a21 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:31.776430 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a869c741-74af-4008-a830-c3a487fcab0e None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:31.798229 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f865bcb0-2f06-42af-9fa4-ade56fa60a21 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:31.820314 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570b67ab-b8eb-43c2-8dc9-b4d0b133ed1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:31.821330 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570b67ab-b8eb-43c2-8dc9-b4d0b133ed1b None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:31.821848 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570b67ab-b8eb-43c2-8dc9-b4d0b133ed1b None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:31.822353 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570b67ab-b8eb-43c2-8dc9-b4d0b133ed1b None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:31.823183 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570b67ab-b8eb-43c2-8dc9-b4d0b133ed1b None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:31.825505 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570b67ab-b8eb-43c2-8dc9-b4d0b133ed1b None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:31.836138 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-570b67ab-b8eb-43c2-8dc9-b4d0b133ed1b None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 06 20:00:31.836138 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:31.836138 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 20:00:31.836138 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 20:00:31.836138 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 20:00:31.836138 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 20:00:31.836138 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 20:00:31.836138 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 20:00:31.836138 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 06 20:00:31.836138 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:00:31.836138 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 20:00:31.836138 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:00:31.836138 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:31.836138 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:00:31.836138 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:00:31.836138 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:00:31.837180 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:00:31.837180 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:00:31.837180 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:00:31.837180 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:00:31.837180 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:00:31.837180 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:00:31.837180 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:00:31.837180 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 20:00:31.837180 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 20:00:31.837180 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 20:00:31.837180 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 20:00:31.837180 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:00:31.837180 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:00:31.837180 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:00:31.837180 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:00:31.837180 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:00:31.837180 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:00:31.838665 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:00:31.838665 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 20:00:31.838665 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:00:31.838665 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:00:31.838665 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:00:31.838665 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:00:31.838665 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:00:31.838665 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:00:31.838665 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:00:31.838665 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:00:31.838665 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 20:00:31.838665 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 20:00:31.838665 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 06 20:00:31.838665 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:00:31.841057 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 455/938] 10.222.0.20 () {56 vars in 1132 bytes} [Tue Sep 6 20:00:31 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:00:31.844952 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebe5c39-dc3f-4939-9da7-e30e3fc3a89d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:31.846160 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebe5c39-dc3f-4939-9da7-e30e3fc3a89d None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:31.846662 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebe5c39-dc3f-4939-9da7-e30e3fc3a89d None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:31.846771 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570b67ab-b8eb-43c2-8dc9-b4d0b133ed1b None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:31.847412 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebe5c39-dc3f-4939-9da7-e30e3fc3a89d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:31.857984 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebe5c39-dc3f-4939-9da7-e30e3fc3a89d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:31.860753 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebe5c39-dc3f-4939-9da7-e30e3fc3a89d None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:31.880867 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-aebe5c39-dc3f-4939-9da7-e30e3fc3a89d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 20:00:31.880867 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:31.880867 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:00:31.880867 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:00:31.880867 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:00:31.880867 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:00:31.880867 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:00:31.880867 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:00:31.880867 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:00:31.880867 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:00:31.880867 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:00:31.880867 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:00:31.880867 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 20:00:31.880867 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 20:00:31.880867 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 20:00:31.881903 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 20:00:31.881903 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:00:31.881903 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:00:31.881903 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:00:31.881903 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:00:31.881903 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:00:31.881903 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:00:31.881903 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:00:31.881903 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 20:00:31.881903 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:00:31.881903 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:00:31.881903 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:00:31.881903 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:00:31.881903 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:00:31.881903 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:00:31.881903 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:00:31.881903 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:00:31.882806 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 20:00:31.882806 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 20:00:31.882806 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 20:00:31.882806 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 20:00:31.882806 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 20:00:31.882806 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 20:00:31.882806 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 20:00:31.882806 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 20:00:31.885732 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 484/939] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 6 20:00:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 20:00:31.891730 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebe5c39-dc3f-4939-9da7-e30e3fc3a89d None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:31.962063 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:31.971061 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:32.001929 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570b67ab-b8eb-43c2-8dc9-b4d0b133ed1b None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:32.026040 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebe5c39-dc3f-4939-9da7-e30e3fc3a89d None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:32.047559 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0669da-4f16-4c8c-bf96-42b85a86c7d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:32.048586 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0669da-4f16-4c8c-bf96-42b85a86c7d6 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:32.049157 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0669da-4f16-4c8c-bf96-42b85a86c7d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:32.049732 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0669da-4f16-4c8c-bf96-42b85a86c7d6 None admin] PATH_INFO: `/v3/services` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:32.050692 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0669da-4f16-4c8c-bf96-42b85a86c7d6 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:32.051217 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0669da-4f16-4c8c-bf96-42b85a86c7d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:32.052959 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0669da-4f16-4c8c-bf96-42b85a86c7d6 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:32.059708 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 456/940] 10.222.0.20 () {56 vars in 1126 bytes} [Tue Sep 6 20:00:31 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:00:32.067359 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0669da-4f16-4c8c-bf96-42b85a86c7d6 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:32.082837 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857b6d45-c820-4637-acd5-b1c80652a16d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:32.084489 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857b6d45-c820-4637-acd5-b1c80652a16d None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:32.085525 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857b6d45-c820-4637-acd5-b1c80652a16d None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:32.086306 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857b6d45-c820-4637-acd5-b1c80652a16d None admin] PATH_INFO: `/v3/projects` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:32.087400 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857b6d45-c820-4637-acd5-b1c80652a16d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:32.087933 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857b6d45-c820-4637-acd5-b1c80652a16d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:32.091279 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857b6d45-c820-4637-acd5-b1c80652a16d None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:32.113900 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 485/941] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Sep 6 20:00:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 20:00:32.118729 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857b6d45-c820-4637-acd5-b1c80652a16d None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:32.221684 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0669da-4f16-4c8c-bf96-42b85a86c7d6 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:32.264138 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98120ae6-130a-487e-8823-4402057e0f83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:32.265286 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98120ae6-130a-487e-8823-4402057e0f83 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:32.265888 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98120ae6-130a-487e-8823-4402057e0f83 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:32.266402 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98120ae6-130a-487e-8823-4402057e0f83 None admin] PATH_INFO: `/v3/services` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:32.267288 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98120ae6-130a-487e-8823-4402057e0f83 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:32.267815 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98120ae6-130a-487e-8823-4402057e0f83 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:32.269691 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98120ae6-130a-487e-8823-4402057e0f83 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:32.276866 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 457/942] 10.222.0.20 () {56 vars in 1126 bytes} [Tue Sep 6 20:00:32 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:00:32.284429 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98120ae6-130a-487e-8823-4402057e0f83 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:32.287861 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:32.307282 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857b6d45-c820-4637-acd5-b1c80652a16d None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:32.329143 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:32.350200 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a7d913-de6a-4eae-94bc-6351f0ac72a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:32.351271 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a7d913-de6a-4eae-94bc-6351f0ac72a0 None admin] REQUEST_METHOD: `PUT` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:32.351415 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a7d913-de6a-4eae-94bc-6351f0ac72a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:32.351603 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a7d913-de6a-4eae-94bc-6351f0ac72a0 None admin] PATH_INFO: `/v3/projects/6aef43493bb1420a9e122313d8db7d5a/users/1a52117df70447dba9bbee35d083e44f/roles/960d5e4b29394e7aae95d5a428dedfc0` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:32.354194 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a7d913-de6a-4eae-94bc-6351f0ac72a0 None admin] RBAC: Authorizing `identity:create_grant(project_id=6aef43493bb1420a9e122313d8db7d5a, user_id=1a52117df70447dba9bbee35d083e44f, role_id=960d5e4b29394e7aae95d5a428dedfc0)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:32.355920 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a7d913-de6a-4eae-94bc-6351f0ac72a0 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:32.385936 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 486/943] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Sep 6 20:00:32 2022] PUT /identity/v3/projects/6aef43493bb1420a9e122313d8db7d5a/users/1a52117df70447dba9bbee35d083e44f/roles/960d5e4b29394e7aae95d5a428dedfc0 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 20:00:32.434314 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98120ae6-130a-487e-8823-4402057e0f83 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:32.484829 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b9ce2b-d046-450c-b6ba-2b0c30b74e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:32.485471 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b9ce2b-d046-450c-b6ba-2b0c30b74e87 None admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:32.485897 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b9ce2b-d046-450c-b6ba-2b0c30b74e87 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:32.486182 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b9ce2b-d046-450c-b6ba-2b0c30b74e87 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:32.487170 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b9ce2b-d046-450c-b6ba-2b0c30b74e87 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:32.488773 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b9ce2b-d046-450c-b6ba-2b0c30b74e87 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:32.521869 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 458/944] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:00:32 2022] POST /identity/v3/endpoints => generated 347 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 20:00:33.238754 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e654309c-6a48-40af-9f59-e94d7ccfff3a None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:33.239581 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e654309c-6a48-40af-9f59-e94d7ccfff3a None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:33.239581 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e654309c-6a48-40af-9f59-e94d7ccfff3a None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:33.241063 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 487/945] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:00:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:00:33.248209 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55994e64-9aac-4c82-a8f7-4d748ebc8bbe None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:33.248699 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55994e64-9aac-4c82-a8f7-4d748ebc8bbe None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:33.249193 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55994e64-9aac-4c82-a8f7-4d748ebc8bbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:33.280478 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-55994e64-9aac-4c82-a8f7-4d748ebc8bbe None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:00:33.282158 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-55994e64-9aac-4c82-a8f7-4d748ebc8bbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:00:33.365056 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:33.405932 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 459/946] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:00:33 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 20:00:33.710033 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e654309c-6a48-40af-9f59-e94d7ccfff3a None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:33.912876 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e654309c-6a48-40af-9f59-e94d7ccfff3a None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:33.984666 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee88322-4215-4120-88d7-525af0833ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:33.986348 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee88322-4215-4120-88d7-525af0833ba5 None admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:33.987179 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee88322-4215-4120-88d7-525af0833ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:33.987973 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee88322-4215-4120-88d7-525af0833ba5 None admin] PATH_INFO: `/v3/projects` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:33.989482 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee88322-4215-4120-88d7-525af0833ba5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:33.992525 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee88322-4215-4120-88d7-525af0833ba5 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:33.997705 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: WARNING keystone.server.flask.common [None req-9ee88322-4215-4120-88d7-525af0833ba5 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 06 20:00:34.031362 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.sql.core [None req-9ee88322-4215-4120-88d7-525af0833ba5 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 06 20:00:34.031362 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 06 20:00:34.031362 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [parameters: {'id': '39152ee39bb04ddeaf0f8c7301c47549', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 06 20:00:34.031362 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81772) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 06 20:00:34.047964 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-9ee88322-4215-4120-88d7-525af0833ba5 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 39152ee39bb04ddeaf0f8c7301c47549.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 39152ee39bb04ddeaf0f8c7301c47549. Sep 06 20:00:34.047964 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:34.047964 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 06 20:00:34.047964 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 06 20:00:34.047964 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 06 20:00:34.047964 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 06 20:00:34.047964 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 06 20:00:34.047964 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application result = self._query(query) Sep 06 20:00:34.047964 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 06 20:00:34.047964 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application conn.query(q) Sep 06 20:00:34.047964 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 06 20:00:34.047964 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 06 20:00:34.047964 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 06 20:00:34.049492 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application result.read() Sep 06 20:00:34.049492 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 06 20:00:34.049492 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 06 20:00:34.049492 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 06 20:00:34.049492 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 06 20:00:34.049492 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 06 20:00:34.049492 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 06 20:00:34.049492 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 06 20:00:34.049492 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 06 20:00:34.049492 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 06 20:00:34.049492 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 20:00:34.049492 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 06 20:00:34.049492 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 20:00:34.049492 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:34.049492 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 06 20:00:34.049492 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 06 20:00:34.049492 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 06 20:00:34.049492 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 06 20:00:34.051111 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 06 20:00:34.051111 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application next(self.gen) Sep 06 20:00:34.051111 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 06 20:00:34.051111 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application yield resource Sep 06 20:00:34.051111 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 06 20:00:34.051111 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application next(self.gen) Sep 06 20:00:34.051111 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 06 20:00:34.051111 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application self.session.rollback() Sep 06 20:00:34.051111 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 06 20:00:34.051111 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application self.force_reraise() Sep 06 20:00:34.051111 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 06 20:00:34.051111 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise self.value Sep 06 20:00:34.051111 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 06 20:00:34.051111 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 06 20:00:34.051111 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 06 20:00:34.051111 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application session.commit() Sep 06 20:00:34.051111 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 06 20:00:34.054628 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 06 20:00:34.054628 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 06 20:00:34.054628 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application self._prepare_impl() Sep 06 20:00:34.054628 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 06 20:00:34.054628 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application self.session.flush() Sep 06 20:00:34.054628 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 06 20:00:34.054628 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application self._flush(objects) Sep 06 20:00:34.054628 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 06 20:00:34.054628 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 06 20:00:34.054628 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 06 20:00:34.054628 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application compat.raise_( Sep 06 20:00:34.054628 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 06 20:00:34.054628 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception Sep 06 20:00:34.054628 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 06 20:00:34.054628 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application flush_context.execute() Sep 06 20:00:34.054628 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 06 20:00:34.054628 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rec.execute(self) Sep 06 20:00:34.058333 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 06 20:00:34.058333 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 06 20:00:34.058333 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 06 20:00:34.058333 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 06 20:00:34.058333 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 06 20:00:34.058333 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 06 20:00:34.058333 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 06 20:00:34.058333 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 06 20:00:34.058333 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 06 20:00:34.058333 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 06 20:00:34.058333 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 06 20:00:34.058333 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 06 20:00:34.058333 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 06 20:00:34.058333 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 06 20:00:34.058333 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 06 20:00:34.061453 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 06 20:00:34.061453 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 06 20:00:34.061453 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception Sep 06 20:00:34.061453 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 06 20:00:34.061453 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 06 20:00:34.061453 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 06 20:00:34.061453 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 06 20:00:34.061453 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 06 20:00:34.061453 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application result = self._query(query) Sep 06 20:00:34.061453 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 06 20:00:34.061453 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application conn.query(q) Sep 06 20:00:34.061453 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 06 20:00:34.061453 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 06 20:00:34.061453 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 06 20:00:34.061453 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application result.read() Sep 06 20:00:34.061453 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 06 20:00:34.061453 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 06 20:00:34.067301 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 06 20:00:34.067301 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 06 20:00:34.067301 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 06 20:00:34.067301 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 06 20:00:34.067301 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 06 20:00:34.067301 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 06 20:00:34.067301 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 06 20:00:34.067301 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 06 20:00:34.067301 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [parameters: {'id': '39152ee39bb04ddeaf0f8c7301c47549', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 06 20:00:34.067301 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 06 20:00:34.067301 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 20:00:34.067301 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 20:00:34.067301 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 20:00:34.067301 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:34.067301 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 06 20:00:34.070227 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 06 20:00:34.070227 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 06 20:00:34.070227 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 06 20:00:34.070227 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 06 20:00:34.070227 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 20:00:34.070227 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 20:00:34.070227 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 20:00:34.070227 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:34.070227 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:00:34.070227 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:00:34.070227 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:00:34.070227 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:00:34.070227 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:00:34.070227 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:00:34.070227 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:00:34.070227 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:00:34.070227 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:00:34.073197 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:00:34.073197 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 06 20:00:34.073197 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 06 20:00:34.073197 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:00:34.073197 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:00:34.073197 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 06 20:00:34.073197 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 06 20:00:34.073197 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 39152ee39bb04ddeaf0f8c7301c47549. Sep 06 20:00:34.073197 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 20:00:34.073197 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 488/947] 10.222.0.20 () {60 vars in 1140 bytes} [Tue Sep 6 20:00:33 2022] POST /identity/v3/projects => generated 262 bytes in 357 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:00:34.075940 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55994e64-9aac-4c82-a8f7-4d748ebc8bbe None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:34.231308 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55994e64-9aac-4c82-a8f7-4d748ebc8bbe None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:34.272312 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44206b54-3873-4ff4-b350-844c8021d00d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:34.272976 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44206b54-3873-4ff4-b350-844c8021d00d None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:34.273257 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44206b54-3873-4ff4-b350-844c8021d00d None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:34.273546 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44206b54-3873-4ff4-b350-844c8021d00d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:34.282023 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44206b54-3873-4ff4-b350-844c8021d00d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:34.284225 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44206b54-3873-4ff4-b350-844c8021d00d None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:34.294818 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfade29a-d8a5-448d-8942-aa9045114e2d None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:34.295587 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfade29a-d8a5-448d-8942-aa9045114e2d None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:34.295723 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfade29a-d8a5-448d-8942-aa9045114e2d None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:34.302525 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-44206b54-3873-4ff4-b350-844c8021d00d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 20:00:34.302525 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:34.302525 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:00:34.302525 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:00:34.302525 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:00:34.302525 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:00:34.302525 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:00:34.302525 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:00:34.302525 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:00:34.302525 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:00:34.302525 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:00:34.302525 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:00:34.302525 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 20:00:34.302525 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 20:00:34.302525 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 20:00:34.304335 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 20:00:34.304335 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:00:34.304335 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:00:34.304335 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:00:34.304335 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:00:34.304335 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:00:34.304335 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:00:34.304335 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:00:34.304335 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 20:00:34.304335 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:00:34.304335 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:00:34.304335 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:00:34.304335 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:00:34.304335 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:00:34.304335 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:00:34.304335 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:00:34.304335 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:00:34.308285 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 20:00:34.308285 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 20:00:34.308285 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 20:00:34.308285 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 20:00:34.308285 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 20:00:34.308285 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 20:00:34.308285 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 20:00:34.308285 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:00:34.310528 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 460/949] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 6 20:00:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:00:34.314207 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44206b54-3873-4ff4-b350-844c8021d00d None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:34.314573 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 489/949] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:00:34 2022] GET /identity => generated 270 bytes in 18 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:00:34.314573 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3902365e-ded6-45e9-800f-354c037b03ef None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:34.314573 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3902365e-ded6-45e9-800f-354c037b03ef None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:34.315381 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3902365e-ded6-45e9-800f-354c037b03ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:34.346737 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-3902365e-ded6-45e9-800f-354c037b03ef None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:00:34.348457 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-3902365e-ded6-45e9-800f-354c037b03ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:00:34.385952 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:34.401647 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:34.477202 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 490/950] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:00:34 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 20:00:34.521785 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44206b54-3873-4ff4-b350-844c8021d00d None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:34.592135 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0ef424-35c3-48e4-9c16-f59814d45c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:34.593222 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0ef424-35c3-48e4-9c16-f59814d45c6b None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:34.594082 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0ef424-35c3-48e4-9c16-f59814d45c6b None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:34.594644 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0ef424-35c3-48e4-9c16-f59814d45c6b None admin] PATH_INFO: `/v3/projects` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:34.595954 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0ef424-35c3-48e4-9c16-f59814d45c6b None admin] RBAC: Adding query filter params (name=service) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:34.596596 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0ef424-35c3-48e4-9c16-f59814d45c6b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:34.599481 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0ef424-35c3-48e4-9c16-f59814d45c6b None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:34.635529 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 461/951] 10.222.0.20 () {56 vars in 1112 bytes} [Tue Sep 6 20:00:34 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:00:34.993118 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3902365e-ded6-45e9-800f-354c037b03ef None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:35.221405 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3902365e-ded6-45e9-800f-354c037b03ef None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:35.233292 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55380edc-19d2-4f09-a595-e3d708729938 None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:35.234094 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55380edc-19d2-4f09-a595-e3d708729938 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:35.234094 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55380edc-19d2-4f09-a595-e3d708729938 None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:35.235983 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 462/952] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:00:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:00:35.243286 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2db4cb-cbfd-4d78-85ee-777eb6dea1bb None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:35.243683 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2db4cb-cbfd-4d78-85ee-777eb6dea1bb None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:35.244182 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2db4cb-cbfd-4d78-85ee-777eb6dea1bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:35.267666 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-5f2db4cb-cbfd-4d78-85ee-777eb6dea1bb None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:00:35.269100 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-5f2db4cb-cbfd-4d78-85ee-777eb6dea1bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:00:35.297602 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dc6859-ef6e-4cd4-8138-0920c9935f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:35.299642 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dc6859-ef6e-4cd4-8138-0920c9935f2f None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:35.300401 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dc6859-ef6e-4cd4-8138-0920c9935f2f None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:35.300956 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dc6859-ef6e-4cd4-8138-0920c9935f2f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:35.302059 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dc6859-ef6e-4cd4-8138-0920c9935f2f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:35.304784 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dc6859-ef6e-4cd4-8138-0920c9935f2f None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:35.321811 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-25dc6859-ef6e-4cd4-8138-0920c9935f2f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 20:00:35.321811 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:35.321811 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 20:00:35.321811 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 20:00:35.321811 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 20:00:35.321811 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 20:00:35.321811 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 20:00:35.321811 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 20:00:35.321811 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 20:00:35.321811 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 20:00:35.321811 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 20:00:35.321811 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 20:00:35.321811 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:35.321811 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:00:35.321811 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:00:35.321811 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:00:35.321811 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:00:35.323930 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:00:35.323930 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:00:35.323930 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:00:35.323930 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:00:35.323930 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:00:35.323930 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:00:35.323930 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 20:00:35.323930 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 20:00:35.323930 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 20:00:35.323930 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 20:00:35.323930 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:00:35.323930 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:00:35.323930 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:00:35.323930 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:00:35.323930 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:00:35.323930 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:00:35.326726 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:00:35.326726 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 20:00:35.326726 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:00:35.326726 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:00:35.326726 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:00:35.326726 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:00:35.326726 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:00:35.326726 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:00:35.326726 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:00:35.326726 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:00:35.326726 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 20:00:35.326726 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 20:00:35.326726 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 20:00:35.326726 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 20:00:35.329773 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 491/953] 10.222.0.20 () {56 vars in 1126 bytes} [Tue Sep 6 20:00:34 2022] GET /identity/v3/services/object-store => generated 93 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:00:35.335097 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dc6859-ef6e-4cd4-8138-0920c9935f2f None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:35.389738 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:35.419477 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 463/954] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:00:35 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 20:00:35.518181 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f1b6f2-e1f1-4b80-9fea-03380305714f None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:35.518418 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f1b6f2-e1f1-4b80-9fea-03380305714f None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:35.518781 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f1b6f2-e1f1-4b80-9fea-03380305714f None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:35.520301 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 464/955] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:00:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 20:00:35.526316 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a080893a-a6f5-4854-87f6-740865048dbb None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:35.526565 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a080893a-a6f5-4854-87f6-740865048dbb None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:35.526939 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a080893a-a6f5-4854-87f6-740865048dbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:35.545507 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dc6859-ef6e-4cd4-8138-0920c9935f2f None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:35.549115 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-a080893a-a6f5-4854-87f6-740865048dbb None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:00:35.550580 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-a080893a-a6f5-4854-87f6-740865048dbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:00:35.595493 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:35.623742 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498d9820-bd30-4172-be4a-c4c5bd498702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:35.624656 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498d9820-bd30-4172-be4a-c4c5bd498702 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:35.625108 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498d9820-bd30-4172-be4a-c4c5bd498702 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:35.625389 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498d9820-bd30-4172-be4a-c4c5bd498702 None admin] PATH_INFO: `/v3/services` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:35.626158 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498d9820-bd30-4172-be4a-c4c5bd498702 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:35.626451 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498d9820-bd30-4172-be4a-c4c5bd498702 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:35.629116 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498d9820-bd30-4172-be4a-c4c5bd498702 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:35.641515 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 492/956] 10.222.0.20 () {56 vars in 1122 bytes} [Tue Sep 6 20:00:35 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 20:00:35.651879 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498d9820-bd30-4172-be4a-c4c5bd498702 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:35.670208 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 465/957] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:00:35 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 20:00:35.799326 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:35.805815 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a080893a-a6f5-4854-87f6-740865048dbb None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:35.816231 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498d9820-bd30-4172-be4a-c4c5bd498702 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:35.882241 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bbfd0a-1ccf-4053-a5fc-d3c056ab56cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:35.883412 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bbfd0a-1ccf-4053-a5fc-d3c056ab56cc None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:35.883869 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bbfd0a-1ccf-4053-a5fc-d3c056ab56cc None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:35.884340 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bbfd0a-1ccf-4053-a5fc-d3c056ab56cc None admin] PATH_INFO: `/v3/services` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:35.885648 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bbfd0a-1ccf-4053-a5fc-d3c056ab56cc None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:35.886182 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bbfd0a-1ccf-4053-a5fc-d3c056ab56cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:35.889515 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bbfd0a-1ccf-4053-a5fc-d3c056ab56cc None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:35.899630 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 493/958] 10.222.0.20 () {56 vars in 1122 bytes} [Tue Sep 6 20:00:35 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:00:35.906691 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bbfd0a-1ccf-4053-a5fc-d3c056ab56cc None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:35.948475 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a080893a-a6f5-4854-87f6-740865048dbb None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:35.994244 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592943b3-4c2d-4195-8d98-b998cd805494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:35.995185 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592943b3-4c2d-4195-8d98-b998cd805494 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:35.995541 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592943b3-4c2d-4195-8d98-b998cd805494 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:35.996510 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592943b3-4c2d-4195-8d98-b998cd805494 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:36.013874 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592943b3-4c2d-4195-8d98-b998cd805494 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 20:00:36.021148 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-592943b3-4c2d-4195-8d98-b998cd805494 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 06 20:00:36.021148 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:36.021148 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:00:36.021148 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:00:36.021148 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:00:36.021148 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:00:36.021148 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:00:36.021148 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:00:36.021148 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:00:36.021148 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:00:36.021148 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:00:36.021148 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:00:36.021148 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 20:00:36.021148 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 20:00:36.021148 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 20:00:36.021148 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 20:00:36.022348 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 20:00:36.022348 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 20:00:36.022348 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 20:00:36.022348 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 20:00:36.022348 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:00:36.022348 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:00:36.022348 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:00:36.022348 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:00:36.022348 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:00:36.022348 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:00:36.022348 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:00:36.022348 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 20:00:36.022348 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:00:36.022348 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:00:36.022348 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:00:36.022348 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:00:36.023202 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:00:36.023202 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:00:36.023202 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:00:36.023202 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:00:36.023202 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 20:00:36.023202 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 20:00:36.023202 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 20:00:36.023202 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 20:00:36.023202 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 20:00:36.023202 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 20:00:36.023202 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 06 20:00:36.023202 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:00:36.024452 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 466/959] 10.222.0.20 () {56 vars in 1120 bytes} [Tue Sep 6 20:00:35 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 20:00:36.026925 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592943b3-4c2d-4195-8d98-b998cd805494 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:36.049252 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bbfd0a-1ccf-4053-a5fc-d3c056ab56cc None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:36.060020 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:36.118263 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6de0271-dcef-4b3e-96fd-a628e420201a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:36.119309 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6de0271-dcef-4b3e-96fd-a628e420201a None admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:36.119808 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6de0271-dcef-4b3e-96fd-a628e420201a None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:36.120169 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6de0271-dcef-4b3e-96fd-a628e420201a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:36.120885 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6de0271-dcef-4b3e-96fd-a628e420201a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:36.123407 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6de0271-dcef-4b3e-96fd-a628e420201a None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:36.161653 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 494/960] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:00:35 2022] POST /identity/v3/endpoints => generated 349 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:00:36.161653 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6de0271-dcef-4b3e-96fd-a628e420201a None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:36.187057 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:36.187360 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592943b3-4c2d-4195-8d98-b998cd805494 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:36.269964 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cb97f0-0f21-43e7-b153-a8143a2a71cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:36.270951 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cb97f0-0f21-43e7-b153-a8143a2a71cc None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:36.271240 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cb97f0-0f21-43e7-b153-a8143a2a71cc None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:36.271801 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cb97f0-0f21-43e7-b153-a8143a2a71cc None admin] PATH_INFO: `/v3/services/compute` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:36.272761 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cb97f0-0f21-43e7-b153-a8143a2a71cc None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:36.274225 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cb97f0-0f21-43e7-b153-a8143a2a71cc None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:36.285115 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-80cb97f0-0f21-43e7-b153-a8143a2a71cc None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 06 20:00:36.285115 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:36.285115 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 20:00:36.285115 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 20:00:36.285115 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 20:00:36.285115 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 20:00:36.285115 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 20:00:36.285115 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 20:00:36.285115 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 06 20:00:36.285115 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:00:36.285115 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 20:00:36.285115 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:00:36.285115 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:36.285115 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:00:36.285115 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:00:36.285115 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:00:36.285115 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:00:36.287293 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:00:36.287293 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:00:36.287293 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:00:36.287293 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:00:36.287293 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:00:36.287293 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:00:36.287293 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 20:00:36.287293 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 20:00:36.287293 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 20:00:36.287293 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 20:00:36.287293 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:00:36.287293 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:00:36.287293 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:00:36.287293 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:00:36.287293 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:00:36.287293 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:00:36.288386 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:00:36.288386 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 20:00:36.288386 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:00:36.288386 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:00:36.288386 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:00:36.288386 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:00:36.288386 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:00:36.288386 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:00:36.288386 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:00:36.288386 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:00:36.288386 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 20:00:36.288386 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 20:00:36.288386 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 06 20:00:36.288386 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:00:36.293927 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 467/961] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 6 20:00:36 2022] GET /identity/v3/services/compute => generated 88 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:00:36.295491 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cb97f0-0f21-43e7-b153-a8143a2a71cc None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:36.334713 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6de0271-dcef-4b3e-96fd-a628e420201a None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:36.381041 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c6dc16-1c35-4615-9512-0a18530d2820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:36.382050 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c6dc16-1c35-4615-9512-0a18530d2820 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:36.382559 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c6dc16-1c35-4615-9512-0a18530d2820 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:36.383001 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c6dc16-1c35-4615-9512-0a18530d2820 None admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:36.384054 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c6dc16-1c35-4615-9512-0a18530d2820 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:36.384729 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c6dc16-1c35-4615-9512-0a18530d2820 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:36.387618 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c6dc16-1c35-4615-9512-0a18530d2820 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:36.406071 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 495/962] 10.222.0.20 () {56 vars in 1115 bytes} [Tue Sep 6 20:00:36 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:00:36.407493 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c6dc16-1c35-4615-9512-0a18530d2820 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:36.423399 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:36.446460 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cb97f0-0f21-43e7-b153-a8143a2a71cc None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:36.492930 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdadc254-83b8-417b-9a0d-cef5e3111f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:36.493601 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdadc254-83b8-417b-9a0d-cef5e3111f92 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:36.493916 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdadc254-83b8-417b-9a0d-cef5e3111f92 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:36.494193 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdadc254-83b8-417b-9a0d-cef5e3111f92 None admin] PATH_INFO: `/v3/services` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:36.494946 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdadc254-83b8-417b-9a0d-cef5e3111f92 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:36.495253 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdadc254-83b8-417b-9a0d-cef5e3111f92 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:36.496916 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdadc254-83b8-417b-9a0d-cef5e3111f92 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:36.505733 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 468/963] 10.222.0.20 () {56 vars in 1112 bytes} [Tue Sep 6 20:00:36 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 20:00:36.510818 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdadc254-83b8-417b-9a0d-cef5e3111f92 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:36.521939 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:36.560510 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c6dc16-1c35-4615-9512-0a18530d2820 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:36.614818 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0026667-20ad-4ee3-a6e8-3e0b079bfdaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:36.615981 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0026667-20ad-4ee3-a6e8-3e0b079bfdaf None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:36.616297 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0026667-20ad-4ee3-a6e8-3e0b079bfdaf None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:36.616579 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0026667-20ad-4ee3-a6e8-3e0b079bfdaf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:36.634578 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0026667-20ad-4ee3-a6e8-3e0b079bfdaf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:36.637972 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0026667-20ad-4ee3-a6e8-3e0b079bfdaf None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:36.659603 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-e0026667-20ad-4ee3-a6e8-3e0b079bfdaf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 20:00:36.659603 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:36.659603 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:00:36.659603 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:00:36.659603 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:00:36.659603 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:00:36.659603 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:00:36.659603 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:00:36.659603 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:00:36.659603 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:00:36.659603 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:00:36.659603 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:00:36.659603 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 20:00:36.659603 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 20:00:36.659603 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 20:00:36.659603 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 20:00:36.660736 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:00:36.660736 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:00:36.660736 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:00:36.660736 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:00:36.660736 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:00:36.660736 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:00:36.660736 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:00:36.660736 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 20:00:36.660736 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:00:36.660736 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:00:36.660736 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:00:36.660736 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:00:36.660736 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:00:36.660736 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:00:36.660736 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:00:36.660736 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:00:36.660736 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 20:00:36.661909 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 20:00:36.661909 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 20:00:36.661909 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 20:00:36.663576 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 496/964] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 20:00:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:00:36.673581 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0026667-20ad-4ee3-a6e8-3e0b079bfdaf None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:36.691965 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdadc254-83b8-417b-9a0d-cef5e3111f92 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:36.740424 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93df2c28-df6c-4179-ad83-13a7eec9f46d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:36.741194 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93df2c28-df6c-4179-ad83-13a7eec9f46d None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:36.741514 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93df2c28-df6c-4179-ad83-13a7eec9f46d None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:36.741822 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93df2c28-df6c-4179-ad83-13a7eec9f46d None admin] PATH_INFO: `/v3/services` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:36.742579 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93df2c28-df6c-4179-ad83-13a7eec9f46d None admin] RBAC: Adding query filter params (type=compute) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:36.742917 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93df2c28-df6c-4179-ad83-13a7eec9f46d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:36.744438 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93df2c28-df6c-4179-ad83-13a7eec9f46d None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:36.753770 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 469/965] 10.222.0.20 () {56 vars in 1112 bytes} [Tue Sep 6 20:00:36 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 20:00:36.875090 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0026667-20ad-4ee3-a6e8-3e0b079bfdaf None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:36.922041 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:36.927343 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d60d323-0c6f-4a98-87e7-cc994e82d61b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:36.927888 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d60d323-0c6f-4a98-87e7-cc994e82d61b None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:36.928303 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d60d323-0c6f-4a98-87e7-cc994e82d61b None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:36.928741 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d60d323-0c6f-4a98-87e7-cc994e82d61b None admin] PATH_INFO: `/v3/domains` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:36.929670 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d60d323-0c6f-4a98-87e7-cc994e82d61b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:36.930002 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d60d323-0c6f-4a98-87e7-cc994e82d61b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:36.931497 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d60d323-0c6f-4a98-87e7-cc994e82d61b None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:36.961651 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 497/966] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 6 20:00:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:00:36.964371 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93df2c28-df6c-4179-ad83-13a7eec9f46d None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:36.985248 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:37.143019 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93df2c28-df6c-4179-ad83-13a7eec9f46d None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:37.212728 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d685de56-806d-484b-ac7d-ecb3003f6742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:37.213872 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d685de56-806d-484b-ac7d-ecb3003f6742 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:37.214369 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d685de56-806d-484b-ac7d-ecb3003f6742 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:37.214936 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d685de56-806d-484b-ac7d-ecb3003f6742 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:37.231184 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d685de56-806d-484b-ac7d-ecb3003f6742 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:37.234098 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d685de56-806d-484b-ac7d-ecb3003f6742 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:37.257600 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-d685de56-806d-484b-ac7d-ecb3003f6742 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 06 20:00:37.257600 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:37.257600 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:00:37.257600 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:00:37.257600 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:00:37.257600 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:00:37.257600 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:00:37.257600 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:00:37.257600 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:00:37.257600 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:00:37.257600 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:00:37.257600 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:00:37.257600 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 20:00:37.257600 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 20:00:37.257600 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 20:00:37.257600 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 20:00:37.258661 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:00:37.258661 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:00:37.258661 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 20:00:37.258661 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 20:00:37.258661 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 20:00:37.258661 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 20:00:37.258661 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:00:37.258661 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:00:37.258661 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:00:37.258661 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:00:37.258661 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:00:37.258661 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 20:00:37.258661 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:00:37.258661 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:00:37.258661 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:00:37.258661 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:00:37.258661 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:00:37.258661 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:00:37.259734 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:00:37.259734 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:00:37.259734 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 20:00:37.259734 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 20:00:37.259734 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 20:00:37.259734 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 20:00:37.259734 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 20:00:37.259734 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 20:00:37.259734 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 06 20:00:37.259734 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:00:37.260314 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 470/967] 10.222.0.20 () {56 vars in 1117 bytes} [Tue Sep 6 20:00:36 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:00:37.269818 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d60d323-0c6f-4a98-87e7-cc994e82d61b None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:37.456383 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d60d323-0c6f-4a98-87e7-cc994e82d61b None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:37.521134 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737cdd5e-6a0f-4ced-9ed8-33ee66e82bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:37.522730 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737cdd5e-6a0f-4ced-9ed8-33ee66e82bb8 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:37.523526 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737cdd5e-6a0f-4ced-9ed8-33ee66e82bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:37.524323 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737cdd5e-6a0f-4ced-9ed8-33ee66e82bb8 None admin] PATH_INFO: `/v3/users` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:37.526088 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737cdd5e-6a0f-4ced-9ed8-33ee66e82bb8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:37.526970 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737cdd5e-6a0f-4ced-9ed8-33ee66e82bb8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:37.530581 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737cdd5e-6a0f-4ced-9ed8-33ee66e82bb8 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:37.560730 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 498/968] 10.222.0.20 () {56 vars in 1149 bytes} [Tue Sep 6 20:00:37 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:00:37.576633 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d685de56-806d-484b-ac7d-ecb3003f6742 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:37.686870 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:37.717050 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d685de56-806d-484b-ac7d-ecb3003f6742 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:37.769601 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8114d83-0ccc-40f8-a29d-dacb5d1f4eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:37.770762 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8114d83-0ccc-40f8-a29d-dacb5d1f4eb8 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:37.771236 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8114d83-0ccc-40f8-a29d-dacb5d1f4eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:37.771679 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8114d83-0ccc-40f8-a29d-dacb5d1f4eb8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:37.906190 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e34f654-70cf-400d-865f-e1096edf4cf6 None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:37.906686 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e34f654-70cf-400d-865f-e1096edf4cf6 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:37.907176 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e34f654-70cf-400d-865f-e1096edf4cf6 None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:37.911423 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 499/969] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:00:37 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:00:37.914016 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8114d83-0ccc-40f8-a29d-dacb5d1f4eb8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:37.915710 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8114d83-0ccc-40f8-a29d-dacb5d1f4eb8 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:37.922489 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b36a4f7-ea79-4cb4-a3ca-853860e7e551 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:37.923195 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b36a4f7-ea79-4cb4-a3ca-853860e7e551 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:37.923697 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b36a4f7-ea79-4cb4-a3ca-853860e7e551 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:37.957763 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-7b36a4f7-ea79-4cb4-a3ca-853860e7e551 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:00:37.961506 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-7b36a4f7-ea79-4cb4-a3ca-853860e7e551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:00:38.024609 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:38.051784 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:38.097268 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 471/970] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 6 20:00:37 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 522 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 20:00:38.105060 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8114d83-0ccc-40f8-a29d-dacb5d1f4eb8 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:38.113812 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 500/971] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:00:37 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 20:00:38.240092 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8114d83-0ccc-40f8-a29d-dacb5d1f4eb8 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:38.285688 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508d8ae2-e160-4283-963c-f9802385d557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:38.286259 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508d8ae2-e160-4283-963c-f9802385d557 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:38.286544 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508d8ae2-e160-4283-963c-f9802385d557 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:38.286814 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508d8ae2-e160-4283-963c-f9802385d557 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:38.299849 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508d8ae2-e160-4283-963c-f9802385d557 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:38.301354 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508d8ae2-e160-4283-963c-f9802385d557 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:38.334670 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-508d8ae2-e160-4283-963c-f9802385d557 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 20:00:38.334670 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:38.334670 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:00:38.334670 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:00:38.334670 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:00:38.334670 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:00:38.334670 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:00:38.334670 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:00:38.334670 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:00:38.334670 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:00:38.334670 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:00:38.334670 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:00:38.334670 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 20:00:38.334670 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 20:00:38.334670 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 20:00:38.334670 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 20:00:38.336573 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:00:38.336573 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:00:38.336573 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:00:38.336573 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:00:38.336573 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:00:38.336573 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:00:38.336573 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:00:38.336573 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 20:00:38.336573 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:00:38.336573 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:00:38.336573 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:00:38.336573 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:00:38.336573 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:00:38.336573 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:00:38.336573 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:00:38.336573 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:00:38.336573 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 20:00:38.340204 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 20:00:38.340204 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 20:00:38.340204 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:00:38.340204 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 472/972] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 20:00:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:00:38.347761 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b36a4f7-ea79-4cb4-a3ca-853860e7e551 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:38.527172 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b36a4f7-ea79-4cb4-a3ca-853860e7e551 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:38.536343 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508d8ae2-e160-4283-963c-f9802385d557 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:38.568202 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628bff4c-38ec-4eb8-9a5c-784918e8540e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:38.568831 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628bff4c-38ec-4eb8-9a5c-784918e8540e None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:38.569116 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628bff4c-38ec-4eb8-9a5c-784918e8540e None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:38.569411 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628bff4c-38ec-4eb8-9a5c-784918e8540e None admin] PATH_INFO: `/v3/domains` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:38.570194 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628bff4c-38ec-4eb8-9a5c-784918e8540e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:38.570501 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628bff4c-38ec-4eb8-9a5c-784918e8540e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:38.572130 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628bff4c-38ec-4eb8-9a5c-784918e8540e None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:38.577008 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:38.591588 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 501/973] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 6 20:00:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:00:38.593063 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628bff4c-38ec-4eb8-9a5c-784918e8540e None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:38.634644 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:38.690681 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508d8ae2-e160-4283-963c-f9802385d557 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:38.721166 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628bff4c-38ec-4eb8-9a5c-784918e8540e None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:38.737676 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26aeb297-da57-4ac7-b2c7-c9c9fe80a9c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:38.738674 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aeb297-da57-4ac7-b2c7-c9c9fe80a9c0 None admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:38.739205 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aeb297-da57-4ac7-b2c7-c9c9fe80a9c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:38.739687 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aeb297-da57-4ac7-b2c7-c9c9fe80a9c0 None admin] PATH_INFO: `/v3/users` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:38.740589 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26aeb297-da57-4ac7-b2c7-c9c9fe80a9c0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:38.742477 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26aeb297-da57-4ac7-b2c7-c9c9fe80a9c0 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:38.745524 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: WARNING keystone.server.flask.common [None req-26aeb297-da57-4ac7-b2c7-c9c9fe80a9c0 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 06 20:00:38.761170 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.sql.core [None req-26aeb297-da57-4ac7-b2c7-c9c9fe80a9c0 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 06 20:00:38.761170 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 06 20:00:38.761170 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [parameters: {'user_id': 'b6b02219e4eb4db6834c932b0c93ea9a', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 06 20:00:38.761170 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81771) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 06 20:00:38.764350 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15afd7b0-b9a4-488d-aac3-2716f696dd05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:38.765012 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15afd7b0-b9a4-488d-aac3-2716f696dd05 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:38.765361 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15afd7b0-b9a4-488d-aac3-2716f696dd05 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:38.765671 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15afd7b0-b9a4-488d-aac3-2716f696dd05 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:38.774443 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15afd7b0-b9a4-488d-aac3-2716f696dd05 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:38.776021 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15afd7b0-b9a4-488d-aac3-2716f696dd05 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:38.781933 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-26aeb297-da57-4ac7-b2c7-c9c9fe80a9c0 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 06 20:00:38.781933 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:38.781933 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 06 20:00:38.781933 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 06 20:00:38.781933 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 06 20:00:38.781933 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 06 20:00:38.781933 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 06 20:00:38.781933 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application result = self._query(query) Sep 06 20:00:38.781933 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 06 20:00:38.781933 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application conn.query(q) Sep 06 20:00:38.781933 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 06 20:00:38.781933 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 06 20:00:38.781933 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 06 20:00:38.781933 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application result.read() Sep 06 20:00:38.781933 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 06 20:00:38.782980 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 06 20:00:38.782980 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 06 20:00:38.782980 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 06 20:00:38.782980 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 06 20:00:38.782980 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 06 20:00:38.782980 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 06 20:00:38.782980 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 06 20:00:38.782980 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 06 20:00:38.782980 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:00:38.782980 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 06 20:00:38.782980 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:00:38.782980 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:38.782980 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 06 20:00:38.782980 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 06 20:00:38.782980 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 06 20:00:38.782980 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 06 20:00:38.782980 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 06 20:00:38.782980 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application next(self.gen) Sep 06 20:00:38.784038 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 06 20:00:38.784038 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application yield resource Sep 06 20:00:38.784038 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 06 20:00:38.784038 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application next(self.gen) Sep 06 20:00:38.784038 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 06 20:00:38.784038 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application self.session.rollback() Sep 06 20:00:38.784038 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 06 20:00:38.784038 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application self.force_reraise() Sep 06 20:00:38.784038 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 06 20:00:38.784038 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise self.value Sep 06 20:00:38.784038 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 06 20:00:38.784038 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 06 20:00:38.784038 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 06 20:00:38.784038 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application session.commit() Sep 06 20:00:38.784038 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 06 20:00:38.784038 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 06 20:00:38.784038 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 06 20:00:38.784971 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application self._prepare_impl() Sep 06 20:00:38.784971 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 06 20:00:38.784971 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application self.session.flush() Sep 06 20:00:38.784971 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 06 20:00:38.784971 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application self._flush(objects) Sep 06 20:00:38.784971 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 06 20:00:38.784971 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 06 20:00:38.784971 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 06 20:00:38.784971 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application compat.raise_( Sep 06 20:00:38.784971 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 06 20:00:38.784971 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception Sep 06 20:00:38.784971 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 06 20:00:38.784971 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application flush_context.execute() Sep 06 20:00:38.784971 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 06 20:00:38.784971 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rec.execute(self) Sep 06 20:00:38.784971 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 06 20:00:38.784971 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 06 20:00:38.786162 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 06 20:00:38.786162 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 06 20:00:38.786162 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 06 20:00:38.786162 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 06 20:00:38.786162 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 06 20:00:38.786162 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 06 20:00:38.786162 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 06 20:00:38.786162 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 06 20:00:38.786162 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 06 20:00:38.786162 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 06 20:00:38.786162 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 06 20:00:38.786162 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 06 20:00:38.786162 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 06 20:00:38.786162 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 06 20:00:38.786162 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 06 20:00:38.786162 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception Sep 06 20:00:38.787288 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 06 20:00:38.787288 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 06 20:00:38.787288 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 06 20:00:38.787288 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 06 20:00:38.787288 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 06 20:00:38.787288 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application result = self._query(query) Sep 06 20:00:38.787288 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 06 20:00:38.787288 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application conn.query(q) Sep 06 20:00:38.787288 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 06 20:00:38.787288 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 06 20:00:38.787288 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 06 20:00:38.787288 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application result.read() Sep 06 20:00:38.787288 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 06 20:00:38.787288 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 06 20:00:38.787288 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 06 20:00:38.787288 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 06 20:00:38.788260 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 06 20:00:38.788260 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 06 20:00:38.788260 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 06 20:00:38.788260 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 06 20:00:38.788260 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 06 20:00:38.788260 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 06 20:00:38.788260 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [parameters: {'user_id': 'b6b02219e4eb4db6834c932b0c93ea9a', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 06 20:00:38.788260 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 06 20:00:38.788260 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:00:38.788260 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 20:00:38.788260 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:00:38.788260 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:38.788260 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:00:38.788260 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:00:38.788260 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:00:38.789208 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:00:38.789208 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:00:38.789208 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:00:38.789208 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:00:38.789208 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:00:38.789208 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:00:38.789208 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:00:38.789208 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 06 20:00:38.789208 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 06 20:00:38.789208 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:00:38.789208 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:00:38.789208 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 20:00:38.789208 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 20:00:38.789208 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 20:00:38.789208 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 20:00:38.789208 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 06 20:00:38.789208 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 06 20:00:38.792306 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 06 20:00:38.792306 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 06 20:00:38.792306 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 06 20:00:38.792306 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 06 20:00:38.792306 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 06 20:00:38.792306 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:00:38.800546 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-15afd7b0-b9a4-488d-aac3-2716f696dd05 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 20:00:38.800546 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:38.800546 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:00:38.800546 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:00:38.800546 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:00:38.800546 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:00:38.800546 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:00:38.800546 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:00:38.800546 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:00:38.800546 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:00:38.800546 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:00:38.800546 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:00:38.800546 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 20:00:38.800546 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 20:00:38.800546 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 20:00:38.802631 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 20:00:38.802631 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:00:38.802631 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:00:38.802631 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:00:38.802631 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:00:38.802631 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:00:38.802631 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:00:38.802631 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:00:38.802631 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 20:00:38.802631 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:00:38.802631 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:00:38.802631 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:00:38.802631 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:00:38.802631 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:00:38.802631 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:00:38.802631 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:00:38.802631 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:00:38.805293 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 20:00:38.805293 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 20:00:38.805293 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 20:00:38.805293 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 20:00:38.805293 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 20:00:38.805293 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 20:00:38.805293 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 20:00:38.805293 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 20:00:38.805293 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 502/974] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 6 20:00:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 20:00:38.805293 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15afd7b0-b9a4-488d-aac3-2716f696dd05 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:38.808759 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 473/975] 10.222.0.20 () {60 vars in 1131 bytes} [Tue Sep 6 20:00:38 2022] POST /identity/v3/users => generated 160 bytes in 269 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:00:38.808759 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26aeb297-da57-4ac7-b2c7-c9c9fe80a9c0 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:38.918782 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:38.929151 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:38.940205 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26aeb297-da57-4ac7-b2c7-c9c9fe80a9c0 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:38.940419 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15afd7b0-b9a4-488d-aac3-2716f696dd05 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:38.986453 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6552d2d-c68b-43ee-8693-40f23bbbbb72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:38.986827 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a59174-ad33-4658-991c-91d588dc0b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:38.987451 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a59174-ad33-4658-991c-91d588dc0b38 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:38.987733 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a59174-ad33-4658-991c-91d588dc0b38 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:38.988086 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a59174-ad33-4658-991c-91d588dc0b38 None admin] PATH_INFO: `/v3/projects` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:38.988441 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6552d2d-c68b-43ee-8693-40f23bbbbb72 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:38.988810 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a59174-ad33-4658-991c-91d588dc0b38 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:38.989126 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a59174-ad33-4658-991c-91d588dc0b38 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:38.989407 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6552d2d-c68b-43ee-8693-40f23bbbbb72 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:38.989990 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6552d2d-c68b-43ee-8693-40f23bbbbb72 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:38.990718 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a59174-ad33-4658-991c-91d588dc0b38 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:39.002578 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6552d2d-c68b-43ee-8693-40f23bbbbb72 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:39.004725 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6552d2d-c68b-43ee-8693-40f23bbbbb72 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:39.013212 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 503/976] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Sep 6 20:00:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 20:00:39.015403 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a59174-ad33-4658-991c-91d588dc0b38 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:39.019730 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-b6552d2d-c68b-43ee-8693-40f23bbbbb72 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 06 20:00:39.019730 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:39.019730 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:00:39.019730 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:00:39.019730 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:00:39.019730 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:00:39.019730 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:00:39.019730 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:00:39.019730 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:00:39.019730 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:00:39.019730 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:00:39.019730 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:00:39.019730 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 20:00:39.019730 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 20:00:39.019730 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 20:00:39.019730 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 20:00:39.020757 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:00:39.020757 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:00:39.020757 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 20:00:39.020757 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 20:00:39.020757 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 20:00:39.020757 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 20:00:39.020757 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:00:39.020757 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:00:39.020757 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:00:39.020757 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:00:39.020757 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:00:39.020757 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 20:00:39.020757 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:00:39.020757 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:00:39.020757 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:00:39.020757 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:00:39.020757 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:00:39.020757 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:00:39.022350 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:00:39.022350 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:00:39.022350 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 20:00:39.022350 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 20:00:39.022350 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 20:00:39.022350 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 20:00:39.022350 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 20:00:39.022350 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 20:00:39.022350 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 06 20:00:39.022350 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:00:39.029711 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 474/977] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 6 20:00:38 2022] GET /identity/v3/users/cinder => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:00:39.032342 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6552d2d-c68b-43ee-8693-40f23bbbbb72 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:39.153909 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a59174-ad33-4658-991c-91d588dc0b38 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:39.164163 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6552d2d-c68b-43ee-8693-40f23bbbbb72 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:39.200993 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3462da-e3b1-4876-accf-fe2ee615b45b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:39.202110 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3462da-e3b1-4876-accf-fe2ee615b45b None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:39.202261 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3462da-e3b1-4876-accf-fe2ee615b45b None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:39.202517 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3462da-e3b1-4876-accf-fe2ee615b45b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:39.203329 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3462da-e3b1-4876-accf-fe2ee615b45b None admin] RBAC: Adding query filter params (role.id=960d5e4b29394e7aae95d5a428dedfc0, scope.project.id=6aef43493bb1420a9e122313d8db7d5a, user.id=1a52117df70447dba9bbee35d083e44f) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:39.203690 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3462da-e3b1-4876-accf-fe2ee615b45b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:39.205871 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3462da-e3b1-4876-accf-fe2ee615b45b None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:39.208686 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f99b5a3-0c94-4d1a-b824-50301357a530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:39.209718 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f99b5a3-0c94-4d1a-b824-50301357a530 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:39.210233 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f99b5a3-0c94-4d1a-b824-50301357a530 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:39.210907 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f99b5a3-0c94-4d1a-b824-50301357a530 None admin] PATH_INFO: `/v3/users` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:39.212118 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f99b5a3-0c94-4d1a-b824-50301357a530 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:39.212676 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f99b5a3-0c94-4d1a-b824-50301357a530 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:39.214691 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f99b5a3-0c94-4d1a-b824-50301357a530 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:39.217889 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 504/978] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Sep 6 20:00:39 2022] GET /identity/v3/role_assignments?user.id=1a52117df70447dba9bbee35d083e44f&scope.project.id=6aef43493bb1420a9e122313d8db7d5a&role.id=960d5e4b29394e7aae95d5a428dedfc0 => generated 608 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 20:00:39.298335 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 475/979] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Sep 6 20:00:39 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 20:00:39.575993 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007ed41e-e3cf-4918-9b9f-0b52f8c5bf74 None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:39.577015 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007ed41e-e3cf-4918-9b9f-0b52f8c5bf74 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:39.577590 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007ed41e-e3cf-4918-9b9f-0b52f8c5bf74 None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:39.579689 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 505/980] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:00:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:00:39.586671 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732786a5-68fb-4e16-bd78-4524ab3e5551 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:39.587637 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732786a5-68fb-4e16-bd78-4524ab3e5551 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:39.588131 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732786a5-68fb-4e16-bd78-4524ab3e5551 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:39.602707 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:39.623733 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-732786a5-68fb-4e16-bd78-4524ab3e5551 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:00:39.627276 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-732786a5-68fb-4e16-bd78-4524ab3e5551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:00:39.772432 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe19e29f-9aba-4270-9b03-6ba77ea57128 None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:39.772864 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe19e29f-9aba-4270-9b03-6ba77ea57128 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:39.773182 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe19e29f-9aba-4270-9b03-6ba77ea57128 None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:39.773931 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 476/981] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:00:39 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 20:00:39.774618 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 506/982] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:00:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 20:00:39.783991 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2c9474-d1f8-4dc4-89ea-af1515f41c60 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:39.784437 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2c9474-d1f8-4dc4-89ea-af1515f41c60 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:39.784835 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2c9474-d1f8-4dc4-89ea-af1515f41c60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:39.807443 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-ca2c9474-d1f8-4dc4-89ea-af1515f41c60 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:00:39.809423 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-ca2c9474-d1f8-4dc4-89ea-af1515f41c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:00:39.896334 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 477/983] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:00:39 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 20:00:40.081955 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe19e29f-9aba-4270-9b03-6ba77ea57128 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:40.127796 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:40.190821 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2c9474-d1f8-4dc4-89ea-af1515f41c60 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:40.259819 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe19e29f-9aba-4270-9b03-6ba77ea57128 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:40.286524 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:40.314271 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820c710a-71b8-45f7-99dd-623af268ea49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:40.315404 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820c710a-71b8-45f7-99dd-623af268ea49 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:40.315892 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820c710a-71b8-45f7-99dd-623af268ea49 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:40.316365 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820c710a-71b8-45f7-99dd-623af268ea49 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:40.317423 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820c710a-71b8-45f7-99dd-623af268ea49 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:40.320059 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820c710a-71b8-45f7-99dd-623af268ea49 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:40.337427 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-820c710a-71b8-45f7-99dd-623af268ea49 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 20:00:40.337427 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:40.337427 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 20:00:40.337427 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 20:00:40.337427 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 20:00:40.337427 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 20:00:40.337427 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 20:00:40.337427 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 20:00:40.337427 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 20:00:40.337427 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 20:00:40.337427 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 20:00:40.337427 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 20:00:40.337427 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:40.337427 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:00:40.337427 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:00:40.337427 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:00:40.337427 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:00:40.340161 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:00:40.340161 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:00:40.340161 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:00:40.340161 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:00:40.340161 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:00:40.340161 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:00:40.340161 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 20:00:40.340161 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 20:00:40.340161 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 20:00:40.340161 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 20:00:40.340161 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:00:40.340161 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:00:40.340161 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:00:40.340161 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:00:40.340161 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:00:40.340161 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:00:40.341967 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:00:40.341967 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 20:00:40.341967 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:00:40.341967 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:00:40.341967 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:00:40.341967 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:00:40.341967 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:00:40.341967 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:00:40.341967 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:00:40.341967 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:00:40.341967 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 20:00:40.341967 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 20:00:40.341967 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 20:00:40.341967 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 20:00:40.341967 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 507/984] 10.222.0.20 () {56 vars in 1126 bytes} [Tue Sep 6 20:00:40 2022] GET /identity/v3/services/object-store => generated 93 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:00:40.347774 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820c710a-71b8-45f7-99dd-623af268ea49 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:40.354700 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2c9474-d1f8-4dc4-89ea-af1515f41c60 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:40.413748 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e880bd-17c0-4d38-8797-c046ec458017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:40.414832 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e880bd-17c0-4d38-8797-c046ec458017 None admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:40.415325 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e880bd-17c0-4d38-8797-c046ec458017 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:40.415822 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e880bd-17c0-4d38-8797-c046ec458017 None admin] PATH_INFO: `/v3/services` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:40.417072 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e880bd-17c0-4d38-8797-c046ec458017 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:40.419933 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e880bd-17c0-4d38-8797-c046ec458017 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:40.453769 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 478/985] 10.222.0.20 () {60 vars in 1141 bytes} [Tue Sep 6 20:00:40 2022] POST /identity/v3/services => generated 242 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 20:00:40.498292 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:40.526717 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820c710a-71b8-45f7-99dd-623af268ea49 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:40.596961 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b41289d-baed-42a4-a8f5-4a706ff31404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:40.597951 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b41289d-baed-42a4-a8f5-4a706ff31404 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:40.597951 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b41289d-baed-42a4-a8f5-4a706ff31404 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:40.598453 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b41289d-baed-42a4-a8f5-4a706ff31404 None admin] PATH_INFO: `/v3/services` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:40.599108 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b41289d-baed-42a4-a8f5-4a706ff31404 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:40.599412 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b41289d-baed-42a4-a8f5-4a706ff31404 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:40.600979 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b41289d-baed-42a4-a8f5-4a706ff31404 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:40.616212 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 508/986] 10.222.0.20 () {56 vars in 1122 bytes} [Tue Sep 6 20:00:40 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:00:40.621550 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e880bd-17c0-4d38-8797-c046ec458017 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:40.804438 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e880bd-17c0-4d38-8797-c046ec458017 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:40.849279 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:40.870317 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30cf313-80ff-4bae-90c3-3fa3ec7ceefe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:40.871453 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30cf313-80ff-4bae-90c3-3fa3ec7ceefe None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:40.871956 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30cf313-80ff-4bae-90c3-3fa3ec7ceefe None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:40.872442 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30cf313-80ff-4bae-90c3-3fa3ec7ceefe None admin] PATH_INFO: `/v3/services` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:40.873541 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30cf313-80ff-4bae-90c3-3fa3ec7ceefe None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:40.873956 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30cf313-80ff-4bae-90c3-3fa3ec7ceefe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:40.876455 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30cf313-80ff-4bae-90c3-3fa3ec7ceefe None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:40.886207 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 479/987] 10.222.0.20 () {56 vars in 1122 bytes} [Tue Sep 6 20:00:40 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:00:40.896413 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b41289d-baed-42a4-a8f5-4a706ff31404 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:41.060742 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b41289d-baed-42a4-a8f5-4a706ff31404 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:41.105445 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e69e01-dcc2-4562-872e-d1f8becb9d52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:41.106282 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e69e01-dcc2-4562-872e-d1f8becb9d52 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:41.106699 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e69e01-dcc2-4562-872e-d1f8becb9d52 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:41.107117 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e69e01-dcc2-4562-872e-d1f8becb9d52 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:41.108053 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e69e01-dcc2-4562-872e-d1f8becb9d52 None admin] RBAC: Adding query filter params (interface=admin, service_id=51bf00ccd5d84eef914a4ece4b88a52e, region_id=RegionOne) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:41.108507 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e69e01-dcc2-4562-872e-d1f8becb9d52 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:41.110177 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e69e01-dcc2-4562-872e-d1f8becb9d52 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:41.122405 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 509/988] 10.222.0.20 () {56 vars in 1249 bytes} [Tue Sep 6 20:00:40 2022] GET /identity/v3/endpoints?service_id=51bf00ccd5d84eef914a4ece4b88a52e&interface=admin®ion_id=RegionOne => generated 193 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:00:41.124234 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30cf313-80ff-4bae-90c3-3fa3ec7ceefe None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:41.256784 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30cf313-80ff-4bae-90c3-3fa3ec7ceefe None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:41.300320 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6be98df-fb8e-4592-8624-4d73246486d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:41.300955 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6be98df-fb8e-4592-8624-4d73246486d5 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:41.301259 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6be98df-fb8e-4592-8624-4d73246486d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:41.301601 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6be98df-fb8e-4592-8624-4d73246486d5 None admin] PATH_INFO: `/v3/services` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:41.302274 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6be98df-fb8e-4592-8624-4d73246486d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:41.303893 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6be98df-fb8e-4592-8624-4d73246486d5 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:41.312244 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 480/989] 10.222.0.20 () {56 vars in 1087 bytes} [Tue Sep 6 20:00:41 2022] GET /identity/v3/services => generated 1727 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 20:00:41.698468 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea38ffc-b4e9-4f57-b0dd-4aaeac2a3971 None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:41.699838 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea38ffc-b4e9-4f57-b0dd-4aaeac2a3971 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:41.700018 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea38ffc-b4e9-4f57-b0dd-4aaeac2a3971 None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:41.701688 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 510/990] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:00:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:00:41.708079 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6b36fa-6010-41d2-9037-937510074e4a None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:41.708876 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6b36fa-6010-41d2-9037-937510074e4a None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:41.709016 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6b36fa-6010-41d2-9037-937510074e4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:41.736923 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-bf6b36fa-6010-41d2-9037-937510074e4a None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:00:41.739026 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-bf6b36fa-6010-41d2-9037-937510074e4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:00:41.832299 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:41.873145 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 481/991] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:00:41 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 20:00:42.160210 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea38ffc-b4e9-4f57-b0dd-4aaeac2a3971 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:42.170989 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:42.303304 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea38ffc-b4e9-4f57-b0dd-4aaeac2a3971 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:42.362895 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b74e1e1-8e48-4954-95be-9b15bc64520b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:42.363959 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b74e1e1-8e48-4954-95be-9b15bc64520b None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:42.364432 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b74e1e1-8e48-4954-95be-9b15bc64520b None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:42.364904 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b74e1e1-8e48-4954-95be-9b15bc64520b None admin] PATH_INFO: `/v3/services/image` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:42.366010 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b74e1e1-8e48-4954-95be-9b15bc64520b None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:42.368578 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b74e1e1-8e48-4954-95be-9b15bc64520b None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:42.387924 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-2b74e1e1-8e48-4954-95be-9b15bc64520b None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 06 20:00:42.387924 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:42.387924 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 20:00:42.387924 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 20:00:42.387924 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 20:00:42.387924 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 20:00:42.387924 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 20:00:42.387924 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 20:00:42.387924 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 06 20:00:42.387924 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 20:00:42.387924 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 20:00:42.387924 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 20:00:42.387924 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:42.387924 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:00:42.387924 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:00:42.387924 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:00:42.387924 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:00:42.389664 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:00:42.389664 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:00:42.389664 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:00:42.389664 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:00:42.389664 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:00:42.389664 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:00:42.389664 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 20:00:42.389664 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 20:00:42.389664 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 20:00:42.389664 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 20:00:42.389664 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:00:42.389664 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:00:42.389664 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:00:42.389664 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:00:42.389664 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:00:42.389664 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:00:42.391968 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:00:42.391968 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 20:00:42.391968 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:00:42.391968 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:00:42.391968 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:00:42.391968 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:00:42.391968 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:00:42.391968 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:00:42.391968 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:00:42.391968 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:00:42.391968 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 20:00:42.391968 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 20:00:42.391968 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 06 20:00:42.391968 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 20:00:42.393171 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 511/992] 10.222.0.20 () {56 vars in 1105 bytes} [Tue Sep 6 20:00:42 2022] GET /identity/v3/services/image => generated 86 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 20:00:42.402033 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6b36fa-6010-41d2-9037-937510074e4a None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:42.598389 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6b36fa-6010-41d2-9037-937510074e4a None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:42.667776 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adc2c4d-a2fc-4a99-b1d2-2687b94b74b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:42.668836 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adc2c4d-a2fc-4a99-b1d2-2687b94b74b3 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:42.669376 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adc2c4d-a2fc-4a99-b1d2-2687b94b74b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:42.669861 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adc2c4d-a2fc-4a99-b1d2-2687b94b74b3 None admin] PATH_INFO: `/v3/services` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:42.671074 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adc2c4d-a2fc-4a99-b1d2-2687b94b74b3 None admin] RBAC: Adding query filter params (name=image) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:42.671623 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adc2c4d-a2fc-4a99-b1d2-2687b94b74b3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:42.674289 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adc2c4d-a2fc-4a99-b1d2-2687b94b74b3 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:42.689199 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 482/993] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 20:00:42 2022] GET /identity/v3/services?name=image => generated 122 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:00:42.691660 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b74e1e1-8e48-4954-95be-9b15bc64520b None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:42.832846 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:42.894865 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b74e1e1-8e48-4954-95be-9b15bc64520b None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:42.959631 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1034eb-12ce-4e54-b702-d36a371a182d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:42.960697 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1034eb-12ce-4e54-b702-d36a371a182d None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:42.961176 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1034eb-12ce-4e54-b702-d36a371a182d None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:42.961705 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1034eb-12ce-4e54-b702-d36a371a182d None admin] PATH_INFO: `/v3/services` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:42.962905 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1034eb-12ce-4e54-b702-d36a371a182d None admin] RBAC: Adding query filter params (type=image) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:42.963451 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1034eb-12ce-4e54-b702-d36a371a182d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:42.966172 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1034eb-12ce-4e54-b702-d36a371a182d None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:42.981664 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 512/994] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 20:00:42 2022] GET /identity/v3/services?type=image => generated 122 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:00:43.822981 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2d7532-9f70-45bd-9f0d-899a30165782 None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:43.823513 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2d7532-9f70-45bd-9f0d-899a30165782 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:43.824371 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2d7532-9f70-45bd-9f0d-899a30165782 None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:43.826943 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 483/995] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:00:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:00:43.833517 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2624175-13fd-4514-bcc5-172edb3a6f2f None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:43.834317 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2624175-13fd-4514-bcc5-172edb3a6f2f None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:43.834317 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2624175-13fd-4514-bcc5-172edb3a6f2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:43.857424 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-f2624175-13fd-4514-bcc5-172edb3a6f2f None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:00:43.858698 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-f2624175-13fd-4514-bcc5-172edb3a6f2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:00:43.953013 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 513/996] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:00:43 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 20:00:44.304917 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4962777c-bb8b-477c-8dbf-7e5576210e18 None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:44.305251 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4962777c-bb8b-477c-8dbf-7e5576210e18 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:44.305696 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4962777c-bb8b-477c-8dbf-7e5576210e18 None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:44.307182 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 484/997] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:00:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:00:44.313760 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3535f5-2b22-49e7-992a-416abae4acaa None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:44.314164 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3535f5-2b22-49e7-992a-416abae4acaa None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:44.314360 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3535f5-2b22-49e7-992a-416abae4acaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:44.338487 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4962777c-bb8b-477c-8dbf-7e5576210e18 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:44.338994 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-3c3535f5-2b22-49e7-992a-416abae4acaa None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:00:44.341675 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-3c3535f5-2b22-49e7-992a-416abae4acaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:00:44.405776 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:44.474994 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:44.492169 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 514/998] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:00:44 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 20:00:44.524164 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4962777c-bb8b-477c-8dbf-7e5576210e18 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:44.586536 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e34503f-10e0-4e6b-b941-23f51e75854d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:44.587252 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e34503f-10e0-4e6b-b941-23f51e75854d None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:44.587524 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e34503f-10e0-4e6b-b941-23f51e75854d None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:44.587834 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e34503f-10e0-4e6b-b941-23f51e75854d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:44.588597 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e34503f-10e0-4e6b-b941-23f51e75854d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:44.590250 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e34503f-10e0-4e6b-b941-23f51e75854d None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:44.601685 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-1e34503f-10e0-4e6b-b941-23f51e75854d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 20:00:44.601685 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:44.601685 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 20:00:44.601685 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 20:00:44.601685 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 20:00:44.601685 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 20:00:44.601685 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 20:00:44.601685 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 20:00:44.601685 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 20:00:44.601685 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:00:44.601685 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 20:00:44.601685 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:00:44.601685 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:44.601685 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:00:44.601685 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:00:44.601685 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:00:44.601685 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:00:44.606337 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:00:44.606337 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:00:44.606337 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:00:44.606337 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:00:44.606337 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:00:44.606337 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:00:44.606337 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 20:00:44.606337 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 20:00:44.606337 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 20:00:44.606337 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 20:00:44.606337 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:00:44.606337 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:00:44.606337 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:00:44.606337 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:00:44.606337 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:00:44.606337 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:00:44.607226 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:00:44.607226 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 20:00:44.607226 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:00:44.607226 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:00:44.607226 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:00:44.607226 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:00:44.607226 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:00:44.607226 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:00:44.607226 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:00:44.607226 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:00:44.607226 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 20:00:44.607226 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 20:00:44.607226 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 20:00:44.607226 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:00:44.607226 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 485/999] 10.222.0.20 () {56 vars in 1126 bytes} [Tue Sep 6 20:00:44 2022] GET /identity/v3/services/object-store => generated 93 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:00:44.614137 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3535f5-2b22-49e7-992a-416abae4acaa None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:44.758347 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3535f5-2b22-49e7-992a-416abae4acaa None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:44.807112 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586bd45d-42c4-40d9-9f61-2f044c2f53ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:44.807916 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586bd45d-42c4-40d9-9f61-2f044c2f53ea None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:44.808407 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586bd45d-42c4-40d9-9f61-2f044c2f53ea None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:44.808879 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586bd45d-42c4-40d9-9f61-2f044c2f53ea None admin] PATH_INFO: `/v3/services` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:44.809896 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586bd45d-42c4-40d9-9f61-2f044c2f53ea None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:44.810394 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586bd45d-42c4-40d9-9f61-2f044c2f53ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:44.812109 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586bd45d-42c4-40d9-9f61-2f044c2f53ea None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:44.825291 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 515/1000] 10.222.0.20 () {56 vars in 1122 bytes} [Tue Sep 6 20:00:44 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:00:44.827128 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e34503f-10e0-4e6b-b941-23f51e75854d None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:44.963069 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586bd45d-42c4-40d9-9f61-2f044c2f53ea None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:44.983281 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e34503f-10e0-4e6b-b941-23f51e75854d None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:44.997837 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:45.037184 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a80905-a179-45d1-9b52-e20f711d7438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:45.038308 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a80905-a179-45d1-9b52-e20f711d7438 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:45.038887 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a80905-a179-45d1-9b52-e20f711d7438 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:45.039396 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a80905-a179-45d1-9b52-e20f711d7438 None admin] PATH_INFO: `/v3/services` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:45.040595 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a80905-a179-45d1-9b52-e20f711d7438 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:45.041127 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a80905-a179-45d1-9b52-e20f711d7438 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:45.043864 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a80905-a179-45d1-9b52-e20f711d7438 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:45.056112 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 486/1001] 10.222.0.20 () {56 vars in 1122 bytes} [Tue Sep 6 20:00:44 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:00:45.063560 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a80905-a179-45d1-9b52-e20f711d7438 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:45.066748 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:45.153889 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586bd45d-42c4-40d9-9f61-2f044c2f53ea None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:45.216891 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a80905-a179-45d1-9b52-e20f711d7438 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:45.225721 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab71e11-886d-4e24-8564-dda400ea4327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:45.226858 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab71e11-886d-4e24-8564-dda400ea4327 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:45.227341 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab71e11-886d-4e24-8564-dda400ea4327 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:45.227809 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab71e11-886d-4e24-8564-dda400ea4327 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:45.228875 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab71e11-886d-4e24-8564-dda400ea4327 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:45.232187 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab71e11-886d-4e24-8564-dda400ea4327 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:45.251182 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-3ab71e11-886d-4e24-8564-dda400ea4327 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 06 20:00:45.251182 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:45.251182 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 20:00:45.251182 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 20:00:45.251182 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 20:00:45.251182 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 20:00:45.251182 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 20:00:45.251182 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 20:00:45.251182 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 06 20:00:45.251182 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 20:00:45.251182 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 20:00:45.251182 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 20:00:45.251182 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:45.251182 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:00:45.251182 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:00:45.251182 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:00:45.251182 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:00:45.253602 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:00:45.253602 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:00:45.253602 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:00:45.253602 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:00:45.253602 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:00:45.253602 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:00:45.253602 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 20:00:45.253602 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 20:00:45.253602 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 20:00:45.253602 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 20:00:45.253602 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:00:45.253602 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:00:45.253602 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:00:45.253602 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:00:45.253602 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:00:45.253602 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:00:45.255342 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:00:45.255342 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 20:00:45.255342 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:00:45.255342 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:00:45.255342 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:00:45.255342 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:00:45.255342 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:00:45.255342 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:00:45.255342 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:00:45.255342 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:00:45.255342 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 20:00:45.255342 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 20:00:45.255342 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 06 20:00:45.255342 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 20:00:45.255342 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 516/1002] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 6 20:00:44 2022] GET /identity/v3/services/compute => generated 88 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 20:00:45.263249 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab71e11-886d-4e24-8564-dda400ea4327 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:45.276217 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e499758-c1c6-4f3f-a9f4-91fbcff34ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:45.277122 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e499758-c1c6-4f3f-a9f4-91fbcff34ee6 None admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:45.277531 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e499758-c1c6-4f3f-a9f4-91fbcff34ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:45.278020 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e499758-c1c6-4f3f-a9f4-91fbcff34ee6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:45.278975 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e499758-c1c6-4f3f-a9f4-91fbcff34ee6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:45.280793 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e499758-c1c6-4f3f-a9f4-91fbcff34ee6 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:45.304992 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 487/1003] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:00:45 2022] POST /identity/v3/endpoints => generated 325 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:00:45.338576 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31ef6a5-f6cb-405c-9e73-25f3c4700267 None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:45.338861 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31ef6a5-f6cb-405c-9e73-25f3c4700267 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:45.339125 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31ef6a5-f6cb-405c-9e73-25f3c4700267 None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:45.340966 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 488/1004] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:00:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 20:00:45.347731 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a4f261-a72b-410c-b37d-0e63533fb925 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:45.348015 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a4f261-a72b-410c-b37d-0e63533fb925 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:45.348258 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a4f261-a72b-410c-b37d-0e63533fb925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:45.378563 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-79a4f261-a72b-410c-b37d-0e63533fb925 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:00:45.379909 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-79a4f261-a72b-410c-b37d-0e63533fb925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:00:45.420590 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:45.484195 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab71e11-886d-4e24-8564-dda400ea4327 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:45.509822 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 489/1005] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:00:45 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 20:00:45.533543 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:45.559818 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1615c84d-f65f-4f0e-aff6-e0b6f9fa9347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:45.559818 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1615c84d-f65f-4f0e-aff6-e0b6f9fa9347 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:45.559818 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1615c84d-f65f-4f0e-aff6-e0b6f9fa9347 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:45.559818 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1615c84d-f65f-4f0e-aff6-e0b6f9fa9347 None admin] PATH_INFO: `/v3/services` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:45.560459 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1615c84d-f65f-4f0e-aff6-e0b6f9fa9347 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:45.560459 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1615c84d-f65f-4f0e-aff6-e0b6f9fa9347 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:45.561271 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1615c84d-f65f-4f0e-aff6-e0b6f9fa9347 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:45.570164 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 517/1006] 10.222.0.20 () {56 vars in 1112 bytes} [Tue Sep 6 20:00:45 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 20:00:45.577069 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a4f261-a72b-410c-b37d-0e63533fb925 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:45.723973 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a4f261-a72b-410c-b37d-0e63533fb925 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:45.767741 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db1437c-1519-429b-8d28-e29c34f886f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:45.768549 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db1437c-1519-429b-8d28-e29c34f886f8 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:45.769127 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db1437c-1519-429b-8d28-e29c34f886f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:45.769913 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db1437c-1519-429b-8d28-e29c34f886f8 None admin] PATH_INFO: `/v3/services` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:45.770881 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db1437c-1519-429b-8d28-e29c34f886f8 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:45.771338 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db1437c-1519-429b-8d28-e29c34f886f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:45.773110 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db1437c-1519-429b-8d28-e29c34f886f8 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:45.785881 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 490/1007] 10.222.0.20 () {56 vars in 1112 bytes} [Tue Sep 6 20:00:45 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:00:45.787462 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1615c84d-f65f-4f0e-aff6-e0b6f9fa9347 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:45.804677 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db1437c-1519-429b-8d28-e29c34f886f8 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:45.852625 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:45.961971 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1615c84d-f65f-4f0e-aff6-e0b6f9fa9347 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:45.990735 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db1437c-1519-429b-8d28-e29c34f886f8 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:46.002897 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9971b346-832f-42d4-8c85-678b0379e881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:46.003587 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9971b346-832f-42d4-8c85-678b0379e881 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:46.003870 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9971b346-832f-42d4-8c85-678b0379e881 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:46.004149 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9971b346-832f-42d4-8c85-678b0379e881 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:46.004954 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9971b346-832f-42d4-8c85-678b0379e881 None admin] RBAC: Adding query filter params (interface=public, service_id=18899fb2ea454bc0a827cbb3e4b92e39, region_id=RegionOne) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:46.005452 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9971b346-832f-42d4-8c85-678b0379e881 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:46.007321 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9971b346-832f-42d4-8c85-678b0379e881 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:46.019837 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 518/1008] 10.222.0.20 () {56 vars in 1251 bytes} [Tue Sep 6 20:00:45 2022] GET /identity/v3/endpoints?service_id=18899fb2ea454bc0a827cbb3e4b92e39&interface=public®ion_id=RegionOne => generated 194 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 20:00:46.021382 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9971b346-832f-42d4-8c85-678b0379e881 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:46.053520 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9eb279-22b7-42d5-9841-ada699ba137e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:46.054431 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9eb279-22b7-42d5-9841-ada699ba137e None admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:46.054862 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9eb279-22b7-42d5-9841-ada699ba137e None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:46.055310 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9eb279-22b7-42d5-9841-ada699ba137e None admin] PATH_INFO: `/v3/services` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:46.056158 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9eb279-22b7-42d5-9841-ada699ba137e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:46.057899 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9eb279-22b7-42d5-9841-ada699ba137e None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:46.077420 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 491/1009] 10.222.0.20 () {60 vars in 1141 bytes} [Tue Sep 6 20:00:45 2022] POST /identity/v3/services => generated 242 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:00:46.118924 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:46.227366 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9971b346-832f-42d4-8c85-678b0379e881 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:46.305279 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edafcea-049f-4146-9769-9f679b4972fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:46.306299 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edafcea-049f-4146-9769-9f679b4972fb None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:46.306672 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edafcea-049f-4146-9769-9f679b4972fb None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:46.306971 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edafcea-049f-4146-9769-9f679b4972fb None admin] PATH_INFO: `/v3/services` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:46.307609 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edafcea-049f-4146-9769-9f679b4972fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:46.309304 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edafcea-049f-4146-9769-9f679b4972fb None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:46.322843 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 519/1010] 10.222.0.20 () {56 vars in 1087 bytes} [Tue Sep 6 20:00:46 2022] GET /identity/v3/services => generated 1957 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:00:48.062972 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69094a7e-176f-453b-b290-f029488822ab None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:48.063814 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69094a7e-176f-453b-b290-f029488822ab None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:48.063814 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69094a7e-176f-453b-b290-f029488822ab None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:48.065560 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 492/1011] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:00:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:00:48.072116 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c060866-a5bc-4c3a-a5ce-687edbe66dc0 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:48.072897 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c060866-a5bc-4c3a-a5ce-687edbe66dc0 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:48.073034 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c060866-a5bc-4c3a-a5ce-687edbe66dc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:48.096486 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-6c060866-a5bc-4c3a-a5ce-687edbe66dc0 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:00:48.097747 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-6c060866-a5bc-4c3a-a5ce-687edbe66dc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:00:48.193036 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 520/1012] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:00:48 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 20:00:48.479383 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69094a7e-176f-453b-b290-f029488822ab None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:48.622898 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:48.657124 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69094a7e-176f-453b-b290-f029488822ab None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:48.704174 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb124a91-07e9-4109-aac5-9f3313c227b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:48.704906 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb124a91-07e9-4109-aac5-9f3313c227b6 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:48.705182 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb124a91-07e9-4109-aac5-9f3313c227b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:48.705465 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb124a91-07e9-4109-aac5-9f3313c227b6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:48.706780 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb124a91-07e9-4109-aac5-9f3313c227b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:48.708324 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb124a91-07e9-4109-aac5-9f3313c227b6 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:48.710424 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 493/1013] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 20:00:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:00:48.718002 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c060866-a5bc-4c3a-a5ce-687edbe66dc0 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:48.804615 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:48.916328 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c060866-a5bc-4c3a-a5ce-687edbe66dc0 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:48.963620 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0719f6-2cc2-45e0-8a5b-4ff9c7dde1cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:48.964298 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0719f6-2cc2-45e0-8a5b-4ff9c7dde1cb None admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:48.964608 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0719f6-2cc2-45e0-8a5b-4ff9c7dde1cb None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:48.964875 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0719f6-2cc2-45e0-8a5b-4ff9c7dde1cb None admin] PATH_INFO: `/v3/projects` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:48.965598 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0719f6-2cc2-45e0-8a5b-4ff9c7dde1cb None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:48.967213 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0719f6-2cc2-45e0-8a5b-4ff9c7dde1cb None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:49.001332 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebea554b-04c3-4862-940a-76242d610aa1 None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:49.001790 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebea554b-04c3-4862-940a-76242d610aa1 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:49.002057 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebea554b-04c3-4862-940a-76242d610aa1 None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:49.003185 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 521/1014] 10.222.0.20 () {60 vars in 1141 bytes} [Tue Sep 6 20:00:48 2022] POST /identity/v3/projects => generated 311 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 20:00:49.003662 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 494/1015] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:00:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 20:00:49.012173 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63065b6f-733c-4905-8585-616a25196d6b None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:49.012445 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63065b6f-733c-4905-8585-616a25196d6b None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:49.012758 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63065b6f-733c-4905-8585-616a25196d6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:49.038198 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-63065b6f-733c-4905-8585-616a25196d6b None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:00:49.039842 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-63065b6f-733c-4905-8585-616a25196d6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:00:49.135773 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 522/1016] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:00:49 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 20:00:49.438125 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebea554b-04c3-4862-940a-76242d610aa1 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:49.473822 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dd4825-cc88-4f4f-8377-b1b7540dc640 None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:49.474307 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dd4825-cc88-4f4f-8377-b1b7540dc640 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:49.474785 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dd4825-cc88-4f4f-8377-b1b7540dc640 None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:49.477676 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 523/1017] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:00:49 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:00:49.484947 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c62968-adea-4af6-a8c7-ab236863b09b None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:49.485556 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c62968-adea-4af6-a8c7-ab236863b09b None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:49.486082 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c62968-adea-4af6-a8c7-ab236863b09b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:49.518403 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-24c62968-adea-4af6-a8c7-ab236863b09b None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:00:49.520691 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-24c62968-adea-4af6-a8c7-ab236863b09b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:00:49.564193 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:49.586256 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebea554b-04c3-4862-940a-76242d610aa1 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:49.637375 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 524/1018] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:00:49 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 20:00:49.637963 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82085fb-567a-4a25-8bb3-f95780ba1441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:49.639023 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82085fb-567a-4a25-8bb3-f95780ba1441 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:49.639491 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82085fb-567a-4a25-8bb3-f95780ba1441 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:49.639958 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82085fb-567a-4a25-8bb3-f95780ba1441 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:49.641058 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82085fb-567a-4a25-8bb3-f95780ba1441 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:49.645014 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82085fb-567a-4a25-8bb3-f95780ba1441 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:49.662974 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-b82085fb-567a-4a25-8bb3-f95780ba1441 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 06 20:00:49.662974 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:49.662974 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 20:00:49.662974 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 20:00:49.662974 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 20:00:49.662974 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 20:00:49.662974 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 20:00:49.662974 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 20:00:49.662974 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 06 20:00:49.662974 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:00:49.662974 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 20:00:49.662974 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:00:49.662974 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:49.662974 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:00:49.662974 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:00:49.662974 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:00:49.662974 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:00:49.665730 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:00:49.665730 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:00:49.665730 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:00:49.665730 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:00:49.665730 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:00:49.665730 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:00:49.665730 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 20:00:49.665730 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 20:00:49.665730 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 20:00:49.665730 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 20:00:49.665730 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:00:49.665730 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:00:49.665730 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:00:49.665730 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:00:49.665730 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:00:49.665730 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:00:49.666992 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:00:49.666992 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 20:00:49.666992 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:00:49.666992 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:00:49.666992 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:00:49.666992 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:00:49.666992 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:00:49.666992 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:00:49.666992 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:00:49.666992 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:00:49.666992 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 20:00:49.666992 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 20:00:49.666992 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 06 20:00:49.666992 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:00:49.666992 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 495/1019] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 6 20:00:49 2022] GET /identity/v3/services/compute => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:00:49.675508 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c62968-adea-4af6-a8c7-ab236863b09b None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:49.846590 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c62968-adea-4af6-a8c7-ab236863b09b None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:49.920852 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bca089-9dea-4c92-b334-b42c532ed64e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:49.921980 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bca089-9dea-4c92-b334-b42c532ed64e None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:49.922458 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bca089-9dea-4c92-b334-b42c532ed64e None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:49.922937 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bca089-9dea-4c92-b334-b42c532ed64e None admin] PATH_INFO: `/v3/services` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:49.924115 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bca089-9dea-4c92-b334-b42c532ed64e None admin] RBAC: Adding query filter params (name=compute) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:49.924697 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bca089-9dea-4c92-b334-b42c532ed64e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:49.927533 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bca089-9dea-4c92-b334-b42c532ed64e None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:49.937342 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 525/1020] 10.222.0.20 () {56 vars in 1112 bytes} [Tue Sep 6 20:00:49 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:00:49.943592 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82085fb-567a-4a25-8bb3-f95780ba1441 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:49.943805 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bca089-9dea-4c92-b334-b42c532ed64e None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:49.969027 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:49.981512 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:50.094201 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bca089-9dea-4c92-b334-b42c532ed64e None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:50.103096 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82085fb-567a-4a25-8bb3-f95780ba1441 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:50.150923 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ef11e8-0582-4988-ada1-0ddb938962c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:50.152548 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ef11e8-0582-4988-ada1-0ddb938962c5 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:50.152886 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ef11e8-0582-4988-ada1-0ddb938962c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:50.153305 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ef11e8-0582-4988-ada1-0ddb938962c5 None admin] PATH_INFO: `/v3/services` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:50.154304 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ef11e8-0582-4988-ada1-0ddb938962c5 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:50.154614 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ef11e8-0582-4988-ada1-0ddb938962c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:50.156212 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ef11e8-0582-4988-ada1-0ddb938962c5 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:50.158747 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90f1173-846b-4c10-8d84-84a9c11ebb3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:50.159691 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90f1173-846b-4c10-8d84-84a9c11ebb3a None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:50.160174 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90f1173-846b-4c10-8d84-84a9c11ebb3a None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:50.160650 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90f1173-846b-4c10-8d84-84a9c11ebb3a None admin] PATH_INFO: `/v3/services/image` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:50.161432 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90f1173-846b-4c10-8d84-84a9c11ebb3a None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:50.163261 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90f1173-846b-4c10-8d84-84a9c11ebb3a None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:50.166617 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 526/1021] 10.222.0.20 () {56 vars in 1112 bytes} [Tue Sep 6 20:00:49 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 20:00:50.170421 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ef11e8-0582-4988-ada1-0ddb938962c5 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:50.177168 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-e90f1173-846b-4c10-8d84-84a9c11ebb3a None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 06 20:00:50.177168 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:50.177168 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 20:00:50.177168 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 20:00:50.177168 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 20:00:50.177168 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 20:00:50.177168 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 20:00:50.177168 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 20:00:50.177168 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 06 20:00:50.177168 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:00:50.177168 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 20:00:50.177168 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:00:50.177168 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:50.177168 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:00:50.177168 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:00:50.177168 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:00:50.177168 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:00:50.178341 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:00:50.178341 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:00:50.178341 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:00:50.178341 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:00:50.178341 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:00:50.178341 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:00:50.178341 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 20:00:50.178341 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 20:00:50.178341 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 20:00:50.178341 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 20:00:50.178341 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:00:50.178341 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:00:50.178341 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:00:50.178341 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:00:50.178341 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:00:50.178341 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:00:50.183852 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:00:50.183852 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 20:00:50.183852 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:00:50.183852 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:00:50.183852 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:00:50.183852 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:00:50.183852 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:00:50.183852 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:00:50.183852 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:00:50.183852 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:00:50.183852 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 20:00:50.183852 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 20:00:50.183852 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 06 20:00:50.183852 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:00:50.183852 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 496/1022] 10.222.0.20 () {56 vars in 1105 bytes} [Tue Sep 6 20:00:49 2022] GET /identity/v3/services/image => generated 86 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:00:50.192127 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90f1173-846b-4c10-8d84-84a9c11ebb3a None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:50.296281 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:50.326060 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ef11e8-0582-4988-ada1-0ddb938962c5 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:50.353259 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:50.381983 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90f1173-846b-4c10-8d84-84a9c11ebb3a None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:50.383978 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4005355e-1872-42d2-aa6e-4cf91f1be0de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:50.384722 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4005355e-1872-42d2-aa6e-4cf91f1be0de None admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:50.385097 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4005355e-1872-42d2-aa6e-4cf91f1be0de None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:50.385377 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4005355e-1872-42d2-aa6e-4cf91f1be0de None admin] PATH_INFO: `/v3/endpoints` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:50.386277 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4005355e-1872-42d2-aa6e-4cf91f1be0de None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:50.387941 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4005355e-1872-42d2-aa6e-4cf91f1be0de None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:50.425660 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 527/1023] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:00:50 2022] POST /identity/v3/endpoints => generated 334 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:00:50.451581 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4ca992-49f3-4d2f-b2a8-5d4718184d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:50.452414 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4ca992-49f3-4d2f-b2a8-5d4718184d25 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:50.452636 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4ca992-49f3-4d2f-b2a8-5d4718184d25 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:50.452807 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4ca992-49f3-4d2f-b2a8-5d4718184d25 None admin] PATH_INFO: `/v3/services` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:50.453396 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4ca992-49f3-4d2f-b2a8-5d4718184d25 None admin] RBAC: Adding query filter params (name=image) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:50.453738 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4ca992-49f3-4d2f-b2a8-5d4718184d25 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:50.455211 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4ca992-49f3-4d2f-b2a8-5d4718184d25 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:50.463381 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 497/1024] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 20:00:50 2022] GET /identity/v3/services?name=image => generated 122 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 20:00:50.469953 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4005355e-1872-42d2-aa6e-4cf91f1be0de None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:50.626004 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4005355e-1872-42d2-aa6e-4cf91f1be0de None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:50.676746 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02130163-20ec-4c5b-85bd-35d24718c20b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:50.677457 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02130163-20ec-4c5b-85bd-35d24718c20b None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:50.677779 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02130163-20ec-4c5b-85bd-35d24718c20b None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:50.678059 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02130163-20ec-4c5b-85bd-35d24718c20b None admin] PATH_INFO: `/v3/services` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:50.678742 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02130163-20ec-4c5b-85bd-35d24718c20b None admin] RBAC: Adding query filter params (type=image) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:50.679049 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02130163-20ec-4c5b-85bd-35d24718c20b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:50.680509 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02130163-20ec-4c5b-85bd-35d24718c20b None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:50.684008 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:50.693828 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 528/1025] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 20:00:50 2022] GET /identity/v3/services?type=image => generated 350 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:00:50.696373 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4ca992-49f3-4d2f-b2a8-5d4718184d25 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:50.837129 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4ca992-49f3-4d2f-b2a8-5d4718184d25 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:50.904516 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6102c39c-2a87-4428-98fa-7ad7a4e2b66a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:50.909170 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6102c39c-2a87-4428-98fa-7ad7a4e2b66a None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:50.909170 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6102c39c-2a87-4428-98fa-7ad7a4e2b66a None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:50.909170 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6102c39c-2a87-4428-98fa-7ad7a4e2b66a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:50.909170 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6102c39c-2a87-4428-98fa-7ad7a4e2b66a None admin] RBAC: Adding query filter params (interface=public, service_id=91b2c4b0625b4cd7bde4535874147339, region_id=RegionOne) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:50.909170 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6102c39c-2a87-4428-98fa-7ad7a4e2b66a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:50.911145 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6102c39c-2a87-4428-98fa-7ad7a4e2b66a None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:50.923285 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 498/1026] 10.222.0.20 () {56 vars in 1251 bytes} [Tue Sep 6 20:00:50 2022] GET /identity/v3/endpoints?service_id=91b2c4b0625b4cd7bde4535874147339&interface=public®ion_id=RegionOne => generated 194 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:00:50.927783 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02130163-20ec-4c5b-85bd-35d24718c20b None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:51.071643 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02130163-20ec-4c5b-85bd-35d24718c20b None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:51.113841 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b174332b-a784-4f7b-bf92-04bc7f4d2dd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:51.114659 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b174332b-a784-4f7b-bf92-04bc7f4d2dd7 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:51.115070 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b174332b-a784-4f7b-bf92-04bc7f4d2dd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:51.115471 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b174332b-a784-4f7b-bf92-04bc7f4d2dd7 None admin] PATH_INFO: `/v3/services` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:51.116232 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b174332b-a784-4f7b-bf92-04bc7f4d2dd7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:51.117860 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b174332b-a784-4f7b-bf92-04bc7f4d2dd7 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:51.132328 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 529/1027] 10.222.0.20 () {56 vars in 1087 bytes} [Tue Sep 6 20:00:50 2022] GET /identity/v3/services => generated 1957 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 20:00:52.503939 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1a558a-c720-4aba-bf53-61096fd2e028 None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:52.504533 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1a558a-c720-4aba-bf53-61096fd2e028 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:52.504895 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1a558a-c720-4aba-bf53-61096fd2e028 None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:52.507620 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 499/1028] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:00:52 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:00:52.517233 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f3c180-3fa8-48ce-9bcb-9583bc8857c2 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:52.517862 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f3c180-3fa8-48ce-9bcb-9583bc8857c2 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:52.518262 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f3c180-3fa8-48ce-9bcb-9583bc8857c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:52.545331 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-06f3c180-3fa8-48ce-9bcb-9583bc8857c2 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:00:52.547040 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-06f3c180-3fa8-48ce-9bcb-9583bc8857c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:00:52.633071 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:52.652660 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 530/1029] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:00:52 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 20:00:52.949962 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1a558a-c720-4aba-bf53-61096fd2e028 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:52.954434 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:53.104219 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027272d6-e8a4-44e0-ba88-a9be3c0958a9 None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:53.104767 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027272d6-e8a4-44e0-ba88-a9be3c0958a9 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:53.105257 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027272d6-e8a4-44e0-ba88-a9be3c0958a9 None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:53.107841 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 531/1030] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:00:53 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:00:53.117758 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5875bbb0-c899-41bb-ad2f-2a8a50e878ff None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:53.118456 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5875bbb0-c899-41bb-ad2f-2a8a50e878ff None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:53.119000 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5875bbb0-c899-41bb-ad2f-2a8a50e878ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:53.131216 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1a558a-c720-4aba-bf53-61096fd2e028 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:53.155316 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-5875bbb0-c899-41bb-ad2f-2a8a50e878ff None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:00:53.157318 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-5875bbb0-c899-41bb-ad2f-2a8a50e878ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:00:53.199784 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f286be38-bafd-41f9-b3fc-5415585885fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:53.201308 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f286be38-bafd-41f9-b3fc-5415585885fd None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:53.201714 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f286be38-bafd-41f9-b3fc-5415585885fd None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:53.202143 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f286be38-bafd-41f9-b3fc-5415585885fd None admin] PATH_INFO: `/v3/domains/default` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:53.204090 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f286be38-bafd-41f9-b3fc-5415585885fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:53.206676 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f286be38-bafd-41f9-b3fc-5415585885fd None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:53.209558 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 500/1031] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 20:00:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 20:00:53.219352 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f286be38-bafd-41f9-b3fc-5415585885fd None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:53.287817 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 532/1032] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:00:53 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 20:00:53.344281 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:53.417844 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f286be38-bafd-41f9-b3fc-5415585885fd None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:53.490978 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7409e63-1431-4512-b96d-d7fc1b9547ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:53.491969 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7409e63-1431-4512-b96d-d7fc1b9547ec None admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:53.492460 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7409e63-1431-4512-b96d-d7fc1b9547ec None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:53.492947 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7409e63-1431-4512-b96d-d7fc1b9547ec None admin] PATH_INFO: `/v3/users` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:53.494251 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7409e63-1431-4512-b96d-d7fc1b9547ec None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:53.497290 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7409e63-1431-4512-b96d-d7fc1b9547ec None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:53.549666 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 501/1033] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Sep 6 20:00:53 2022] POST /identity/v3/users => generated 285 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 20:00:53.607616 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f14142-1c75-4213-a17e-7b45ac59da46 None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:53.608654 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f14142-1c75-4213-a17e-7b45ac59da46 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:53.608654 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f14142-1c75-4213-a17e-7b45ac59da46 None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:53.611179 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 533/1034] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:00:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 20:00:53.619824 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec20657f-06c9-4bcc-89eb-7a01180038d2 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:53.620464 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec20657f-06c9-4bcc-89eb-7a01180038d2 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:53.620772 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec20657f-06c9-4bcc-89eb-7a01180038d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:53.627636 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f14142-1c75-4213-a17e-7b45ac59da46 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:53.651713 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-ec20657f-06c9-4bcc-89eb-7a01180038d2 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:00:53.653763 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-ec20657f-06c9-4bcc-89eb-7a01180038d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:00:53.793730 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 502/1035] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:00:53 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 20:00:53.800621 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f14142-1c75-4213-a17e-7b45ac59da46 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:53.835330 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:53.851162 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8055cc-42b0-45d1-9c5a-157588dc983d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:53.851999 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8055cc-42b0-45d1-9c5a-157588dc983d None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:53.852405 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8055cc-42b0-45d1-9c5a-157588dc983d None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:53.852814 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8055cc-42b0-45d1-9c5a-157588dc983d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:53.869048 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8055cc-42b0-45d1-9c5a-157588dc983d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 20:00:53.876403 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-bd8055cc-42b0-45d1-9c5a-157588dc983d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 06 20:00:53.876403 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:53.876403 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:00:53.876403 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:00:53.876403 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:00:53.876403 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:00:53.876403 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:00:53.876403 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:00:53.876403 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:00:53.876403 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:00:53.876403 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:00:53.876403 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:00:53.876403 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 20:00:53.876403 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 20:00:53.876403 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 20:00:53.876403 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 20:00:53.878725 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 20:00:53.878725 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 20:00:53.878725 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 20:00:53.878725 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 20:00:53.878725 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:00:53.878725 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:00:53.878725 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:00:53.878725 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:00:53.878725 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:00:53.878725 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:00:53.878725 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:00:53.878725 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 20:00:53.878725 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:00:53.878725 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:00:53.878725 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:00:53.878725 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:00:53.880462 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:00:53.880462 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:00:53.880462 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:00:53.880462 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:00:53.880462 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 20:00:53.880462 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 20:00:53.880462 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 20:00:53.880462 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 20:00:53.880462 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 20:00:53.880462 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 20:00:53.880462 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 06 20:00:53.880462 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 20:00:53.883516 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 534/1036] 10.222.0.20 () {56 vars in 1120 bytes} [Tue Sep 6 20:00:53 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 20:00:53.885599 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec20657f-06c9-4bcc-89eb-7a01180038d2 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:54.015526 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:54.100641 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8055cc-42b0-45d1-9c5a-157588dc983d None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:54.108127 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec20657f-06c9-4bcc-89eb-7a01180038d2 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:54.174279 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf522c9-cf6d-45aa-966b-75319898d4c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:54.175270 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf522c9-cf6d-45aa-966b-75319898d4c4 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:54.175723 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf522c9-cf6d-45aa-966b-75319898d4c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:54.176186 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf522c9-cf6d-45aa-966b-75319898d4c4 None admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:54.177409 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf522c9-cf6d-45aa-966b-75319898d4c4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:54.177963 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf522c9-cf6d-45aa-966b-75319898d4c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:54.180545 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf522c9-cf6d-45aa-966b-75319898d4c4 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:54.197470 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 503/1037] 10.222.0.20 () {56 vars in 1115 bytes} [Tue Sep 6 20:00:53 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:00:54.206077 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf522c9-cf6d-45aa-966b-75319898d4c4 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:54.263598 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8055cc-42b0-45d1-9c5a-157588dc983d None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:54.318411 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e02fb5c-7cc7-4ab5-8d29-52fac34a8d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:54.319069 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e02fb5c-7cc7-4ab5-8d29-52fac34a8d4a None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:54.319381 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e02fb5c-7cc7-4ab5-8d29-52fac34a8d4a None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:54.319647 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e02fb5c-7cc7-4ab5-8d29-52fac34a8d4a None admin] PATH_INFO: `/v3/services/image` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:54.320222 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e02fb5c-7cc7-4ab5-8d29-52fac34a8d4a None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:54.321850 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e02fb5c-7cc7-4ab5-8d29-52fac34a8d4a None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:54.333065 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-4e02fb5c-7cc7-4ab5-8d29-52fac34a8d4a None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 06 20:00:54.333065 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:54.333065 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 20:00:54.333065 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 20:00:54.333065 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 20:00:54.333065 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 20:00:54.333065 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 20:00:54.333065 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 20:00:54.333065 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 06 20:00:54.333065 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 20:00:54.333065 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 20:00:54.333065 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 20:00:54.333065 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:54.333065 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:00:54.333065 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:00:54.333065 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:00:54.333065 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:00:54.335229 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:00:54.335229 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:00:54.335229 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:00:54.335229 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:00:54.335229 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:00:54.335229 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:00:54.335229 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 20:00:54.335229 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 20:00:54.335229 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 20:00:54.335229 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 20:00:54.335229 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:00:54.335229 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:00:54.335229 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:00:54.335229 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:00:54.335229 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:00:54.335229 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:00:54.338646 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:00:54.338646 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 20:00:54.338646 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:00:54.338646 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:00:54.338646 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:00:54.338646 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:00:54.338646 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:00:54.338646 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:00:54.338646 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:00:54.338646 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:00:54.338646 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 20:00:54.338646 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 20:00:54.338646 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 06 20:00:54.338646 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 20:00:54.338646 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 535/1038] 10.222.0.20 () {56 vars in 1105 bytes} [Tue Sep 6 20:00:54 2022] GET /identity/v3/services/image => generated 86 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 20:00:54.344623 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e02fb5c-7cc7-4ab5-8d29-52fac34a8d4a None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:54.392492 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf522c9-cf6d-45aa-966b-75319898d4c4 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:54.432699 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:54.450929 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:54.460856 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7462fd8-7586-4cc0-aea4-b326f2593989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:54.462066 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7462fd8-7586-4cc0-aea4-b326f2593989 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:54.462594 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7462fd8-7586-4cc0-aea4-b326f2593989 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:54.463046 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7462fd8-7586-4cc0-aea4-b326f2593989 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:54.483533 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7462fd8-7586-4cc0-aea4-b326f2593989 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:54.485997 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7462fd8-7586-4cc0-aea4-b326f2593989 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:54.514680 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-a7462fd8-7586-4cc0-aea4-b326f2593989 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 20:00:54.514680 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:54.514680 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:00:54.514680 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:00:54.514680 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:00:54.514680 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:00:54.514680 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:00:54.514680 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:00:54.514680 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:00:54.514680 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:00:54.514680 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:00:54.514680 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:00:54.514680 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 20:00:54.514680 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 20:00:54.514680 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 20:00:54.514680 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 20:00:54.516614 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:00:54.516614 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:00:54.516614 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:00:54.516614 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:00:54.516614 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:00:54.516614 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:00:54.516614 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:00:54.516614 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 20:00:54.516614 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:00:54.516614 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:00:54.516614 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:00:54.516614 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:00:54.516614 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:00:54.516614 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:00:54.516614 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:00:54.516614 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:00:54.516614 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 20:00:54.518675 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 20:00:54.518675 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 20:00:54.518675 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:00:54.518675 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 504/1039] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 20:00:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:00:54.524246 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7462fd8-7586-4cc0-aea4-b326f2593989 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:54.566063 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e02fb5c-7cc7-4ab5-8d29-52fac34a8d4a None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:54.643684 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b555736d-e4d1-448d-9166-ad58a47d7162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:54.643684 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b555736d-e4d1-448d-9166-ad58a47d7162 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:54.643684 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b555736d-e4d1-448d-9166-ad58a47d7162 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:54.643684 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b555736d-e4d1-448d-9166-ad58a47d7162 None admin] PATH_INFO: `/v3/services` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:54.645212 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b555736d-e4d1-448d-9166-ad58a47d7162 None admin] RBAC: Adding query filter params (name=image) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:54.645403 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b555736d-e4d1-448d-9166-ad58a47d7162 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:54.647978 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b555736d-e4d1-448d-9166-ad58a47d7162 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:54.660148 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 536/1040] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 20:00:54 2022] GET /identity/v3/services?name=image => generated 122 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 20:00:54.667348 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b555736d-e4d1-448d-9166-ad58a47d7162 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:54.709770 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7462fd8-7586-4cc0-aea4-b326f2593989 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:54.780957 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d32cd5e-ca60-4f2a-afb9-0e0bfca59a97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:54.782107 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d32cd5e-ca60-4f2a-afb9-0e0bfca59a97 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:54.782609 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d32cd5e-ca60-4f2a-afb9-0e0bfca59a97 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:54.783128 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d32cd5e-ca60-4f2a-afb9-0e0bfca59a97 None admin] PATH_INFO: `/v3/domains` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:54.784466 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d32cd5e-ca60-4f2a-afb9-0e0bfca59a97 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:54.785069 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d32cd5e-ca60-4f2a-afb9-0e0bfca59a97 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:54.787948 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d32cd5e-ca60-4f2a-afb9-0e0bfca59a97 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:54.820204 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 505/1041] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 6 20:00:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 20:00:54.821815 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d32cd5e-ca60-4f2a-afb9-0e0bfca59a97 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:54.866231 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b555736d-e4d1-448d-9166-ad58a47d7162 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:54.884721 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:54.889160 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:54.932636 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf2f321-b352-49d5-b502-45ec32ab8492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:54.934404 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf2f321-b352-49d5-b502-45ec32ab8492 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:54.934852 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf2f321-b352-49d5-b502-45ec32ab8492 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:54.935297 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf2f321-b352-49d5-b502-45ec32ab8492 None admin] PATH_INFO: `/v3/services` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:54.936426 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf2f321-b352-49d5-b502-45ec32ab8492 None admin] RBAC: Adding query filter params (type=image) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:54.936932 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf2f321-b352-49d5-b502-45ec32ab8492 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:54.939201 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf2f321-b352-49d5-b502-45ec32ab8492 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:54.949710 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 537/1042] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 20:00:54 2022] GET /identity/v3/services?type=image => generated 350 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 20:00:54.958072 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf2f321-b352-49d5-b502-45ec32ab8492 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:55.033982 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d32cd5e-ca60-4f2a-afb9-0e0bfca59a97 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:55.084169 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a6e5c4-a9a4-49dd-9b6a-5291be4d8755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:55.085011 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a6e5c4-a9a4-49dd-9b6a-5291be4d8755 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:55.085481 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a6e5c4-a9a4-49dd-9b6a-5291be4d8755 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:55.085979 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a6e5c4-a9a4-49dd-9b6a-5291be4d8755 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:55.096055 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a6e5c4-a9a4-49dd-9b6a-5291be4d8755 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:55.097905 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a6e5c4-a9a4-49dd-9b6a-5291be4d8755 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:55.114152 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-87a6e5c4-a9a4-49dd-9b6a-5291be4d8755 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 06 20:00:55.114152 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:55.114152 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:00:55.114152 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:00:55.114152 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:00:55.114152 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:00:55.114152 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:00:55.114152 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:00:55.114152 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:00:55.114152 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:00:55.114152 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:00:55.114152 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:00:55.114152 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 20:00:55.114152 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 20:00:55.114152 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 20:00:55.114152 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 20:00:55.116109 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:00:55.116109 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:00:55.116109 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 20:00:55.116109 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 20:00:55.116109 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 20:00:55.116109 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 20:00:55.116109 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:00:55.116109 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:00:55.116109 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:00:55.116109 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:00:55.116109 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:00:55.116109 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 20:00:55.116109 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:00:55.116109 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:00:55.116109 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:00:55.116109 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:00:55.116109 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:00:55.116109 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:00:55.117224 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:00:55.117224 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:00:55.117224 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 20:00:55.117224 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 20:00:55.117224 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 20:00:55.117224 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 20:00:55.117224 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 20:00:55.117224 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 20:00:55.117224 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 06 20:00:55.117224 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:00:55.117224 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 506/1043] 10.222.0.20 () {56 vars in 1093 bytes} [Tue Sep 6 20:00:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 20:00:55.126330 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a6e5c4-a9a4-49dd-9b6a-5291be4d8755 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:55.152469 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf2f321-b352-49d5-b502-45ec32ab8492 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:55.223337 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afc8bc4-5037-4cd5-989a-4d1b7aa039ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:55.223862 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afc8bc4-5037-4cd5-989a-4d1b7aa039ee None admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:55.224149 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afc8bc4-5037-4cd5-989a-4d1b7aa039ee None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:55.224643 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afc8bc4-5037-4cd5-989a-4d1b7aa039ee None admin] PATH_INFO: `/v3/endpoints` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:55.226051 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afc8bc4-5037-4cd5-989a-4d1b7aa039ee None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:55.227984 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afc8bc4-5037-4cd5-989a-4d1b7aa039ee None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:55.276195 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 538/1044] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:00:54 2022] POST /identity/v3/endpoints => generated 327 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:00:55.276535 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:55.295475 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a6e5c4-a9a4-49dd-9b6a-5291be4d8755 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:55.353928 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d216a9-3e51-4b37-a36d-4ec66dd7955b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:55.354666 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d216a9-3e51-4b37-a36d-4ec66dd7955b None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:55.355022 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d216a9-3e51-4b37-a36d-4ec66dd7955b None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:55.355302 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d216a9-3e51-4b37-a36d-4ec66dd7955b None admin] PATH_INFO: `/v3/users` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:55.356104 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d216a9-3e51-4b37-a36d-4ec66dd7955b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:55.356558 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d216a9-3e51-4b37-a36d-4ec66dd7955b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:55.358283 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d216a9-3e51-4b37-a36d-4ec66dd7955b None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:55.384136 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 507/1045] 10.222.0.20 () {56 vars in 1133 bytes} [Tue Sep 6 20:00:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 20:00:55.391453 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afc8bc4-5037-4cd5-989a-4d1b7aa039ee None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:55.437047 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:55.549107 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afc8bc4-5037-4cd5-989a-4d1b7aa039ee None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:55.596446 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66515e52-293d-48ed-98a2-07c7535757de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:55.597457 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66515e52-293d-48ed-98a2-07c7535757de None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:55.597940 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66515e52-293d-48ed-98a2-07c7535757de None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:55.598428 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66515e52-293d-48ed-98a2-07c7535757de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:55.720763 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66515e52-293d-48ed-98a2-07c7535757de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:55.724694 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66515e52-293d-48ed-98a2-07c7535757de None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:55.785508 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:55.931418 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 539/1046] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 6 20:00:55 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 540 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 20:00:55.940934 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d216a9-3e51-4b37-a36d-4ec66dd7955b None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:56.140720 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d216a9-3e51-4b37-a36d-4ec66dd7955b None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:56.192352 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bef604d-a583-4d5d-b9de-ee0ff23790cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:56.193089 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bef604d-a583-4d5d-b9de-ee0ff23790cb None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:56.193548 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bef604d-a583-4d5d-b9de-ee0ff23790cb None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:56.194074 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bef604d-a583-4d5d-b9de-ee0ff23790cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:56.211468 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bef604d-a583-4d5d-b9de-ee0ff23790cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:56.213060 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bef604d-a583-4d5d-b9de-ee0ff23790cb None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:56.225061 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:56.241579 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-4bef604d-a583-4d5d-b9de-ee0ff23790cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 20:00:56.241579 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:56.241579 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:00:56.241579 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:00:56.241579 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:00:56.241579 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:00:56.241579 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:00:56.241579 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:00:56.241579 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:00:56.241579 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:00:56.241579 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:00:56.241579 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:00:56.241579 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 20:00:56.241579 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 20:00:56.241579 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 20:00:56.241579 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 20:00:56.243358 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:00:56.243358 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:00:56.243358 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:00:56.243358 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:00:56.243358 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:00:56.243358 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:00:56.243358 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:00:56.243358 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 20:00:56.243358 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:00:56.243358 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:00:56.243358 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:00:56.243358 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:00:56.243358 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:00:56.243358 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:00:56.243358 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:00:56.243358 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:00:56.243358 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 20:00:56.245260 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 20:00:56.245260 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 20:00:56.245260 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:00:56.247770 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 508/1047] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 20:00:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:00:56.250489 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66515e52-293d-48ed-98a2-07c7535757de None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:56.384317 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66515e52-293d-48ed-98a2-07c7535757de None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:56.428681 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:56.438294 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e811971-2f9f-4377-bbf0-393b996ee121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:56.438933 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e811971-2f9f-4377-bbf0-393b996ee121 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:56.439220 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e811971-2f9f-4377-bbf0-393b996ee121 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:56.439485 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e811971-2f9f-4377-bbf0-393b996ee121 None admin] PATH_INFO: `/v3/domains` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:56.440160 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e811971-2f9f-4377-bbf0-393b996ee121 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:56.440467 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e811971-2f9f-4377-bbf0-393b996ee121 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:56.442109 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e811971-2f9f-4377-bbf0-393b996ee121 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:56.457717 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 540/1048] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 6 20:00:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 20:00:56.464821 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bef604d-a583-4d5d-b9de-ee0ff23790cb None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:56.630981 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bef604d-a583-4d5d-b9de-ee0ff23790cb None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:56.696798 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1276eff9-15be-4b08-8d26-b9cd1f8291dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:56.697672 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1276eff9-15be-4b08-8d26-b9cd1f8291dc None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:56.697982 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1276eff9-15be-4b08-8d26-b9cd1f8291dc None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:56.698256 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1276eff9-15be-4b08-8d26-b9cd1f8291dc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:56.707944 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1276eff9-15be-4b08-8d26-b9cd1f8291dc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:56.709784 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1276eff9-15be-4b08-8d26-b9cd1f8291dc None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:56.725134 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-1276eff9-15be-4b08-8d26-b9cd1f8291dc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 20:00:56.725134 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:56.725134 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:00:56.725134 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:00:56.725134 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:00:56.725134 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:00:56.725134 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:00:56.725134 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:00:56.725134 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:00:56.725134 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:00:56.725134 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:00:56.725134 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:00:56.725134 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 20:00:56.725134 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 20:00:56.725134 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 20:00:56.726948 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 20:00:56.726948 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:00:56.726948 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:00:56.726948 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:00:56.726948 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:00:56.726948 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:00:56.726948 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:00:56.726948 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:00:56.726948 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 20:00:56.726948 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:00:56.726948 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:00:56.726948 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:00:56.726948 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:00:56.726948 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:00:56.726948 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:00:56.726948 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:00:56.726948 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:00:56.728847 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 20:00:56.728847 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 20:00:56.728847 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 20:00:56.728847 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 20:00:56.728847 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 20:00:56.728847 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 20:00:56.728847 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 20:00:56.728847 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:00:56.733758 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 509/1049] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 6 20:00:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 20:00:56.738882 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e811971-2f9f-4377-bbf0-393b996ee121 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:56.954656 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e811971-2f9f-4377-bbf0-393b996ee121 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:56.984839 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2075d974-2f34-42d7-b9d8-dde3cf72fb0d None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:56.985581 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2075d974-2f34-42d7-b9d8-dde3cf72fb0d None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:56.985581 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2075d974-2f34-42d7-b9d8-dde3cf72fb0d None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:56.988345 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 510/1050] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:00:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:00:56.998024 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edb5733-a12a-4262-93aa-da17e2820929 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:56.998526 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edb5733-a12a-4262-93aa-da17e2820929 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:56.999002 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edb5733-a12a-4262-93aa-da17e2820929 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:57.027385 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-2edb5733-a12a-4262-93aa-da17e2820929 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:00:57.029104 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-2edb5733-a12a-4262-93aa-da17e2820929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:00:57.031146 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2198a75-c052-4a17-962c-7f78dff745a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:57.031450 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2198a75-c052-4a17-962c-7f78dff745a3 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:57.032244 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2198a75-c052-4a17-962c-7f78dff745a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:57.032956 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2198a75-c052-4a17-962c-7f78dff745a3 None admin] PATH_INFO: `/v3/projects` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:57.034726 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2198a75-c052-4a17-962c-7f78dff745a3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:57.034993 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2198a75-c052-4a17-962c-7f78dff745a3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:57.037013 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2198a75-c052-4a17-962c-7f78dff745a3 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:57.056517 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 541/1051] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Sep 6 20:00:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:00:57.065113 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2198a75-c052-4a17-962c-7f78dff745a3 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:57.146762 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:57.160794 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:57.169699 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 511/1052] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:00:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 20:00:57.277124 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2198a75-c052-4a17-962c-7f78dff745a3 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:57.361674 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcf0ede-f297-488c-bb8e-d423f09a71cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:57.362733 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcf0ede-f297-488c-bb8e-d423f09a71cd None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:57.363227 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcf0ede-f297-488c-bb8e-d423f09a71cd None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:57.363708 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcf0ede-f297-488c-bb8e-d423f09a71cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:57.365156 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcf0ede-f297-488c-bb8e-d423f09a71cd None admin] RBAC: Adding query filter params (role.id=960d5e4b29394e7aae95d5a428dedfc0, scope.project.id=6aef43493bb1420a9e122313d8db7d5a, user.id=5776cbf4f6714b2083c3ebdd6d98cc87) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:57.365746 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcf0ede-f297-488c-bb8e-d423f09a71cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:57.368914 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcf0ede-f297-488c-bb8e-d423f09a71cd None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:57.387873 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 542/1053] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Sep 6 20:00:57 2022] GET /identity/v3/role_assignments?user.id=5776cbf4f6714b2083c3ebdd6d98cc87&scope.project.id=6aef43493bb1420a9e122313d8db7d5a&role.id=960d5e4b29394e7aae95d5a428dedfc0 => generated 259 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:00:57.584947 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edb5733-a12a-4262-93aa-da17e2820929 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:57.802729 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edb5733-a12a-4262-93aa-da17e2820929 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:57.877723 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12d342c-9bb8-4dc7-8403-1e0bfdcf4954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:57.879602 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12d342c-9bb8-4dc7-8403-1e0bfdcf4954 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:57.880644 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12d342c-9bb8-4dc7-8403-1e0bfdcf4954 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:57.881564 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12d342c-9bb8-4dc7-8403-1e0bfdcf4954 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:57.904836 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12d342c-9bb8-4dc7-8403-1e0bfdcf4954 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 20:00:57.915207 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-a12d342c-9bb8-4dc7-8403-1e0bfdcf4954 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 20:00:57.915207 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:57.915207 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:00:57.915207 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:00:57.915207 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:00:57.915207 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:00:57.915207 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:00:57.915207 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:00:57.915207 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:00:57.915207 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:00:57.915207 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:00:57.915207 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:00:57.915207 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 20:00:57.915207 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 20:00:57.915207 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 20:00:57.915207 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 20:00:57.917214 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 20:00:57.917214 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 20:00:57.917214 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 20:00:57.917214 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 20:00:57.917214 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:00:57.917214 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:00:57.917214 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:00:57.917214 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:00:57.917214 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:00:57.917214 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:00:57.917214 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:00:57.917214 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 20:00:57.917214 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:00:57.917214 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:00:57.917214 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:00:57.917214 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:00:57.919730 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:00:57.919730 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:00:57.919730 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:00:57.919730 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:00:57.919730 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 20:00:57.919730 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 20:00:57.919730 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 20:00:57.919730 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 20:00:57.919730 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 20:00:57.919730 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 20:00:57.919730 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 20:00:57.919730 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:00:57.919730 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 512/1054] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Sep 6 20:00:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:00:57.925295 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcf0ede-f297-488c-bb8e-d423f09a71cd None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:58.076907 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967ace31-8697-4110-bcc8-f6eee44df591 None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:58.077766 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967ace31-8697-4110-bcc8-f6eee44df591 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:58.077766 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967ace31-8697-4110-bcc8-f6eee44df591 None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:58.081100 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 513/1055] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:00:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:00:58.083828 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcf0ede-f297-488c-bb8e-d423f09a71cd None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:58.088442 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11b53da-5923-4516-b784-6a3dda51b022 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:58.089107 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11b53da-5923-4516-b784-6a3dda51b022 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:58.089703 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11b53da-5923-4516-b784-6a3dda51b022 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:58.110033 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:58.122555 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-c11b53da-5923-4516-b784-6a3dda51b022 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:00:58.124853 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-c11b53da-5923-4516-b784-6a3dda51b022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:00:58.137383 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bab9e06-7af4-4657-a7ac-92d8ab80677e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:58.137945 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bab9e06-7af4-4657-a7ac-92d8ab80677e None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:58.138225 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bab9e06-7af4-4657-a7ac-92d8ab80677e None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:58.138500 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bab9e06-7af4-4657-a7ac-92d8ab80677e None admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:58.139198 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bab9e06-7af4-4657-a7ac-92d8ab80677e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:58.139499 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bab9e06-7af4-4657-a7ac-92d8ab80677e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:58.141817 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bab9e06-7af4-4657-a7ac-92d8ab80677e None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:58.156859 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 543/1056] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 6 20:00:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:00:58.162407 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bab9e06-7af4-4657-a7ac-92d8ab80677e None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:58.205976 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:58.235323 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 514/1057] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:00:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 20:00:58.355305 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bab9e06-7af4-4657-a7ac-92d8ab80677e None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:58.425464 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d47fdf-5ce8-445d-abdb-5ee19906bbdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:58.426673 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d47fdf-5ce8-445d-abdb-5ee19906bbdd None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:58.427164 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d47fdf-5ce8-445d-abdb-5ee19906bbdd None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:58.427651 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d47fdf-5ce8-445d-abdb-5ee19906bbdd None admin] PATH_INFO: `/v3/users/cd152c34d6ec447fb7ba9f421f6bffc7` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:58.456365 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d47fdf-5ce8-445d-abdb-5ee19906bbdd None admin] RBAC: Authorizing `identity:get_user(user_id=cd152c34d6ec447fb7ba9f421f6bffc7)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:58.459283 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d47fdf-5ce8-445d-abdb-5ee19906bbdd None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:58.462006 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 544/1058] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 6 20:00:58 2022] GET /identity/v3/users/cd152c34d6ec447fb7ba9f421f6bffc7 => generated 285 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:00:58.468983 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11b53da-5923-4516-b784-6a3dda51b022 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:58.562729 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d47fdf-5ce8-445d-abdb-5ee19906bbdd None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:58.612242 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:58.624153 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11b53da-5923-4516-b784-6a3dda51b022 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:58.688647 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02692625-fb9c-4eb3-9f85-e85fe69e27b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:58.689228 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02692625-fb9c-4eb3-9f85-e85fe69e27b0 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:58.689642 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02692625-fb9c-4eb3-9f85-e85fe69e27b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:58.689898 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02692625-fb9c-4eb3-9f85-e85fe69e27b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:58.710639 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d47fdf-5ce8-445d-abdb-5ee19906bbdd None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:58.759778 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:58.762820 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148e2c56-cc76-4781-b7aa-7ef709ee5c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:58.763620 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148e2c56-cc76-4781-b7aa-7ef709ee5c02 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:58.763824 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148e2c56-cc76-4781-b7aa-7ef709ee5c02 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:58.764076 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148e2c56-cc76-4781-b7aa-7ef709ee5c02 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:58.794420 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02692625-fb9c-4eb3-9f85-e85fe69e27b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:58.797767 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02692625-fb9c-4eb3-9f85-e85fe69e27b0 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:58.871879 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148e2c56-cc76-4781-b7aa-7ef709ee5c02 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:58.874009 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148e2c56-cc76-4781-b7aa-7ef709ee5c02 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:58.930982 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:58.957912 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 515/1059] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 6 20:00:58 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 06 20:00:58.961970 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02692625-fb9c-4eb3-9f85-e85fe69e27b0 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:59.044193 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 545/1060] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 6 20:00:58 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 06 20:00:59.045540 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148e2c56-cc76-4781-b7aa-7ef709ee5c02 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:59.097503 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:59.139947 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02692625-fb9c-4eb3-9f85-e85fe69e27b0 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:59.186931 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f135cf-edb6-46e5-961e-92b5dab75d97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:59.187742 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f135cf-edb6-46e5-961e-92b5dab75d97 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:59.187903 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f135cf-edb6-46e5-961e-92b5dab75d97 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:59.188190 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f135cf-edb6-46e5-961e-92b5dab75d97 None admin] PATH_INFO: `/v3/projects/da8eb4b247e14d629bb0e825fe00b39d` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:59.189521 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f135cf-edb6-46e5-961e-92b5dab75d97 None admin] RBAC: Authorizing `identity:get_project(project_id=da8eb4b247e14d629bb0e825fe00b39d)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:59.191058 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f135cf-edb6-46e5-961e-92b5dab75d97 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:59.193090 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 516/1061] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 6 20:00:58 2022] GET /identity/v3/projects/da8eb4b247e14d629bb0e825fe00b39d => generated 311 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:00:59.201233 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f135cf-edb6-46e5-961e-92b5dab75d97 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:59.232292 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148e2c56-cc76-4781-b7aa-7ef709ee5c02 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:59.299729 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711780a0-4697-4d69-b643-2dcd59f43711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:59.300791 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711780a0-4697-4d69-b643-2dcd59f43711 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:59.301278 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711780a0-4697-4d69-b643-2dcd59f43711 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:59.301903 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711780a0-4697-4d69-b643-2dcd59f43711 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:59.306857 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:00:59.321297 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711780a0-4697-4d69-b643-2dcd59f43711 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:59.323253 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711780a0-4697-4d69-b643-2dcd59f43711 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:59.342492 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f135cf-edb6-46e5-961e-92b5dab75d97 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:59.349001 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-711780a0-4697-4d69-b643-2dcd59f43711 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 20:00:59.349001 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:00:59.349001 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:00:59.349001 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:00:59.349001 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:00:59.349001 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:00:59.349001 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:00:59.349001 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:00:59.349001 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:00:59.349001 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:00:59.349001 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:00:59.349001 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:00:59.349001 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 20:00:59.349001 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 20:00:59.349001 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 20:00:59.349001 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 20:00:59.350414 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:00:59.350414 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:00:59.350414 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:00:59.350414 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:00:59.350414 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:00:59.350414 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:00:59.350414 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:00:59.350414 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 20:00:59.350414 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:00:59.350414 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:00:59.350414 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:00:59.350414 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:00:59.350414 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:00:59.350414 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:00:59.350414 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:00:59.350414 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:00:59.350414 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 20:00:59.351737 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 20:00:59.351737 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 20:00:59.351737 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 20:00:59.353997 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 546/1062] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 20:00:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:00:59.360793 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711780a0-4697-4d69-b643-2dcd59f43711 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:00:59.395206 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2913fcd0-cec6-4be0-99ba-4f86bf571954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:59.396097 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2913fcd0-cec6-4be0-99ba-4f86bf571954 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:59.396569 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2913fcd0-cec6-4be0-99ba-4f86bf571954 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:59.396981 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2913fcd0-cec6-4be0-99ba-4f86bf571954 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:59.398275 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2913fcd0-cec6-4be0-99ba-4f86bf571954 None admin] RBAC: Adding query filter params (role.id=e0ab41d8ae22474688b5361bcc75e0f0, scope.project.id=da8eb4b247e14d629bb0e825fe00b39d, user.id=cd152c34d6ec447fb7ba9f421f6bffc7) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:59.398724 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2913fcd0-cec6-4be0-99ba-4f86bf571954 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:59.400991 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2913fcd0-cec6-4be0-99ba-4f86bf571954 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:59.417960 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 517/1063] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Sep 6 20:00:59 2022] GET /identity/v3/role_assignments?user.id=cd152c34d6ec447fb7ba9f421f6bffc7&scope.project.id=da8eb4b247e14d629bb0e825fe00b39d&role.id=e0ab41d8ae22474688b5361bcc75e0f0 => generated 259 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 20:00:59.508208 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711780a0-4697-4d69-b643-2dcd59f43711 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:00:59.564955 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdb9aa1-d9f2-4585-b6b9-82283a8cf8d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:00:59.565803 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdb9aa1-d9f2-4585-b6b9-82283a8cf8d2 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:00:59.566081 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdb9aa1-d9f2-4585-b6b9-82283a8cf8d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:00:59.566352 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdb9aa1-d9f2-4585-b6b9-82283a8cf8d2 None admin] PATH_INFO: `/v3/domains` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:00:59.567129 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdb9aa1-d9f2-4585-b6b9-82283a8cf8d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:00:59.567439 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdb9aa1-d9f2-4585-b6b9-82283a8cf8d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:00:59.569141 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdb9aa1-d9f2-4585-b6b9-82283a8cf8d2 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:00:59.593670 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 547/1064] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 6 20:00:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 20:01:01.049830 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd1857b-8a5a-41b7-9dcf-63d6ac24900a None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:01.050444 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd1857b-8a5a-41b7-9dcf-63d6ac24900a None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:01.050789 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd1857b-8a5a-41b7-9dcf-63d6ac24900a None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:01.053608 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 518/1065] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:01:01 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:01:01.062630 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7fb0b3-3f93-4f2e-83dd-491fa015b5bf None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:01.063476 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7fb0b3-3f93-4f2e-83dd-491fa015b5bf None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:01.063721 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7fb0b3-3f93-4f2e-83dd-491fa015b5bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:01.092116 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-af7fb0b3-3f93-4f2e-83dd-491fa015b5bf None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:01:01.094214 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-af7fb0b3-3f93-4f2e-83dd-491fa015b5bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:01:01.158070 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:01:01.188375 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 548/1066] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:01:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 20:01:01.479125 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd1857b-8a5a-41b7-9dcf-63d6ac24900a None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:01:01.496905 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:01:01.671172 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd1857b-8a5a-41b7-9dcf-63d6ac24900a None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:01:01.715868 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31320f91-903d-4383-bf23-5979851e7f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:01:01.716413 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31320f91-903d-4383-bf23-5979851e7f4d None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:01.716691 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31320f91-903d-4383-bf23-5979851e7f4d None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:01.717006 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31320f91-903d-4383-bf23-5979851e7f4d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:01.731649 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31320f91-903d-4383-bf23-5979851e7f4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 20:01:01.737077 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-31320f91-903d-4383-bf23-5979851e7f4d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 06 20:01:01.737077 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:01:01.737077 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:01:01.737077 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:01:01.737077 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:01:01.737077 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:01:01.737077 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:01:01.737077 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:01:01.737077 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:01:01.737077 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:01:01.737077 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:01:01.737077 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:01:01.737077 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 20:01:01.737077 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 20:01:01.737077 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 20:01:01.737077 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 20:01:01.739001 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 20:01:01.739001 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 20:01:01.739001 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 20:01:01.739001 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 20:01:01.739001 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:01:01.739001 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:01:01.739001 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:01:01.739001 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:01:01.739001 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:01:01.739001 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:01:01.739001 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:01:01.739001 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 20:01:01.739001 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:01:01.739001 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:01:01.739001 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:01:01.739001 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:01:01.740630 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:01:01.740630 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:01:01.740630 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:01:01.740630 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:01:01.740630 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 20:01:01.740630 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 20:01:01.740630 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 20:01:01.740630 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 20:01:01.740630 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 20:01:01.740630 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 20:01:01.740630 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 06 20:01:01.740630 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:01:01.740630 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 519/1067] 10.222.0.20 () {56 vars in 1120 bytes} [Tue Sep 6 20:01:01 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 20:01:01.746820 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7fb0b3-3f93-4f2e-83dd-491fa015b5bf None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:01:01.938778 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d5b2ac-f249-4cfd-95ce-a02a0c530df2 None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:01.939090 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d5b2ac-f249-4cfd-95ce-a02a0c530df2 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:01.939337 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d5b2ac-f249-4cfd-95ce-a02a0c530df2 None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:01.946579 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 520/1068] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:01:01 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 20:01:01.948033 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de75d72-a42f-4260-a366-1e93eac5fb06 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:01.948317 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de75d72-a42f-4260-a366-1e93eac5fb06 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:01.949189 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de75d72-a42f-4260-a366-1e93eac5fb06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:01.960379 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7fb0b3-3f93-4f2e-83dd-491fa015b5bf None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:01:01.980547 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-4de75d72-a42f-4260-a366-1e93eac5fb06 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:01:01.982137 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-4de75d72-a42f-4260-a366-1e93eac5fb06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:01:02.034285 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7d9333-8c9a-4a30-b070-a6cb5f129740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:01:02.035518 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7d9333-8c9a-4a30-b070-a6cb5f129740 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:02.036002 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7d9333-8c9a-4a30-b070-a6cb5f129740 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:02.036471 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7d9333-8c9a-4a30-b070-a6cb5f129740 None admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:02.037737 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7d9333-8c9a-4a30-b070-a6cb5f129740 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:01:02.038261 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7d9333-8c9a-4a30-b070-a6cb5f129740 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:01:02.040798 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7d9333-8c9a-4a30-b070-a6cb5f129740 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:01:02.057688 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 549/1069] 10.222.0.20 () {56 vars in 1115 bytes} [Tue Sep 6 20:01:01 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:01:02.065132 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7d9333-8c9a-4a30-b070-a6cb5f129740 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:01:02.083072 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:01:02.093834 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 521/1070] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:01:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 20:01:02.144763 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:01:02.225779 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7d9333-8c9a-4a30-b070-a6cb5f129740 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:01:02.278663 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42733090-a32c-46e3-9fda-9d9df13f0711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:01:02.279874 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42733090-a32c-46e3-9fda-9d9df13f0711 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:02.280422 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42733090-a32c-46e3-9fda-9d9df13f0711 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:02.280969 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42733090-a32c-46e3-9fda-9d9df13f0711 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:02.302621 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42733090-a32c-46e3-9fda-9d9df13f0711 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:01:02.305218 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42733090-a32c-46e3-9fda-9d9df13f0711 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:01:02.329108 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-42733090-a32c-46e3-9fda-9d9df13f0711 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 20:01:02.329108 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:01:02.329108 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:01:02.329108 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:01:02.329108 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:01:02.329108 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:01:02.329108 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:01:02.329108 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:01:02.329108 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:01:02.329108 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:01:02.329108 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:01:02.329108 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:01:02.329108 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 20:01:02.329108 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 20:01:02.329108 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 20:01:02.329108 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 20:01:02.338757 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:01:02.338757 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:01:02.338757 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:01:02.338757 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:01:02.338757 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:01:02.338757 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:01:02.338757 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:01:02.338757 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 20:01:02.338757 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:01:02.338757 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:01:02.338757 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:01:02.338757 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:01:02.338757 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:01:02.338757 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:01:02.338757 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:01:02.338757 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:01:02.338757 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 20:01:02.340950 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 20:01:02.340950 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 20:01:02.340950 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 20:01:02.340950 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 550/1071] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 20:01:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:01:02.341425 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de75d72-a42f-4260-a366-1e93eac5fb06 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:01:02.427714 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42733090-a32c-46e3-9fda-9d9df13f0711 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:01:02.479323 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de75d72-a42f-4260-a366-1e93eac5fb06 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:01:02.544235 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438d3295-6f34-4ed3-8b63-8f01aad90e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:01:02.545171 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438d3295-6f34-4ed3-8b63-8f01aad90e27 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:02.545681 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438d3295-6f34-4ed3-8b63-8f01aad90e27 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:02.546161 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438d3295-6f34-4ed3-8b63-8f01aad90e27 None admin] PATH_INFO: `/v3/domains` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:02.547299 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438d3295-6f34-4ed3-8b63-8f01aad90e27 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:01:02.547793 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438d3295-6f34-4ed3-8b63-8f01aad90e27 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:01:02.550282 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438d3295-6f34-4ed3-8b63-8f01aad90e27 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:01:02.556001 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:01:02.571537 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42733090-a32c-46e3-9fda-9d9df13f0711 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:01:02.572899 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 522/1072] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 6 20:01:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 20:01:02.584444 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438d3295-6f34-4ed3-8b63-8f01aad90e27 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:01:02.633989 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b66527-245e-4584-898a-c04af86b87fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:01:02.634771 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b66527-245e-4584-898a-c04af86b87fd None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:02.635151 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b66527-245e-4584-898a-c04af86b87fd None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:02.635520 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b66527-245e-4584-898a-c04af86b87fd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:02.652846 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b66527-245e-4584-898a-c04af86b87fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 20:01:02.658062 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-57b66527-245e-4584-898a-c04af86b87fd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 20:01:02.658062 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:01:02.658062 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:01:02.658062 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:01:02.658062 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:01:02.658062 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:01:02.658062 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:01:02.658062 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:01:02.658062 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:01:02.658062 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:01:02.658062 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:01:02.658062 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:01:02.658062 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 20:01:02.658062 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 20:01:02.658062 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 20:01:02.658062 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 20:01:02.660401 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 20:01:02.660401 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 20:01:02.660401 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 20:01:02.660401 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 20:01:02.660401 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:01:02.660401 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:01:02.660401 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:01:02.660401 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:01:02.660401 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:01:02.660401 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:01:02.660401 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:01:02.660401 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 20:01:02.660401 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:01:02.660401 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:01:02.660401 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:01:02.660401 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:01:02.661535 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:01:02.661535 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:01:02.661535 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:01:02.661535 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:01:02.661535 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 20:01:02.661535 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 20:01:02.661535 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 20:01:02.661535 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 20:01:02.661535 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 20:01:02.661535 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 20:01:02.661535 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 20:01:02.661535 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 20:01:02.661535 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 551/1073] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Sep 6 20:01:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 20:01:02.670064 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b66527-245e-4584-898a-c04af86b87fd None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:01:02.682622 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:01:02.741950 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438d3295-6f34-4ed3-8b63-8f01aad90e27 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:01:02.795050 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156d2ac9-e8f0-4e16-bf14-a5f712b68a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:01:02.795945 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156d2ac9-e8f0-4e16-bf14-a5f712b68a40 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:02.796428 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156d2ac9-e8f0-4e16-bf14-a5f712b68a40 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:02.796881 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156d2ac9-e8f0-4e16-bf14-a5f712b68a40 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:02.810171 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156d2ac9-e8f0-4e16-bf14-a5f712b68a40 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:01:02.812656 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156d2ac9-e8f0-4e16-bf14-a5f712b68a40 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:01:02.834115 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-156d2ac9-e8f0-4e16-bf14-a5f712b68a40 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 06 20:01:02.834115 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:01:02.834115 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:01:02.834115 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:01:02.834115 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:01:02.834115 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:01:02.834115 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:01:02.834115 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:01:02.834115 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:01:02.834115 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:01:02.834115 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:01:02.834115 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:01:02.834115 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 20:01:02.834115 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 20:01:02.834115 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 20:01:02.834115 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 20:01:02.836082 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:01:02.836082 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:01:02.836082 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 20:01:02.836082 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 20:01:02.836082 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 20:01:02.836082 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 20:01:02.836082 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:01:02.836082 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:01:02.836082 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:01:02.836082 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:01:02.836082 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:01:02.836082 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 20:01:02.836082 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:01:02.836082 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:01:02.836082 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:01:02.836082 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:01:02.836082 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:01:02.836082 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:01:02.838108 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:01:02.838108 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:01:02.838108 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 20:01:02.838108 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 20:01:02.838108 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 20:01:02.838108 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 20:01:02.838108 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 20:01:02.838108 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 20:01:02.838108 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 06 20:01:02.838108 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:01:02.841840 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 523/1074] 10.222.0.20 () {56 vars in 1093 bytes} [Tue Sep 6 20:01:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 20:01:02.845964 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156d2ac9-e8f0-4e16-bf14-a5f712b68a40 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:01:02.847929 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b66527-245e-4584-898a-c04af86b87fd None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:01:02.908875 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e429493e-dffa-471c-b3cb-7836caded334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:01:02.909883 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e429493e-dffa-471c-b3cb-7836caded334 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:02.910339 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e429493e-dffa-471c-b3cb-7836caded334 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:02.910783 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e429493e-dffa-471c-b3cb-7836caded334 None admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:02.911910 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e429493e-dffa-471c-b3cb-7836caded334 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:01:02.912465 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e429493e-dffa-471c-b3cb-7836caded334 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:01:02.915136 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e429493e-dffa-471c-b3cb-7836caded334 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:01:02.930731 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 552/1075] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 6 20:01:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 20:01:02.940036 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e429493e-dffa-471c-b3cb-7836caded334 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:01:02.963206 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:01:03.012621 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156d2ac9-e8f0-4e16-bf14-a5f712b68a40 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:01:03.040110 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:01:03.060478 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00619e4c-9a20-46f3-b3dc-f1839840e08c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:01:03.061401 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00619e4c-9a20-46f3-b3dc-f1839840e08c None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:03.061935 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00619e4c-9a20-46f3-b3dc-f1839840e08c None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:03.062416 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00619e4c-9a20-46f3-b3dc-f1839840e08c None admin] PATH_INFO: `/v3/users` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:03.063925 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00619e4c-9a20-46f3-b3dc-f1839840e08c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:01:03.064461 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00619e4c-9a20-46f3-b3dc-f1839840e08c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:01:03.067126 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00619e4c-9a20-46f3-b3dc-f1839840e08c None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:01:03.107421 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 524/1076] 10.222.0.20 () {56 vars in 1133 bytes} [Tue Sep 6 20:01:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 20:01:03.108853 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00619e4c-9a20-46f3-b3dc-f1839840e08c None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:01:03.109224 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e429493e-dffa-471c-b3cb-7836caded334 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:01:03.165279 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa7ce18-7e4a-40e7-bad1-4c162010e570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:01:03.166509 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa7ce18-7e4a-40e7-bad1-4c162010e570 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:03.166955 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa7ce18-7e4a-40e7-bad1-4c162010e570 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:03.167520 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa7ce18-7e4a-40e7-bad1-4c162010e570 None admin] PATH_INFO: `/v3/users/cd152c34d6ec447fb7ba9f421f6bffc7` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:03.169857 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa7ce18-7e4a-40e7-bad1-4c162010e570 None admin] RBAC: Authorizing `identity:get_user(user_id=cd152c34d6ec447fb7ba9f421f6bffc7)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:01:03.172652 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa7ce18-7e4a-40e7-bad1-4c162010e570 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:01:03.175989 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 553/1077] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 6 20:01:02 2022] GET /identity/v3/users/cd152c34d6ec447fb7ba9f421f6bffc7 => generated 285 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 20:01:03.184419 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa7ce18-7e4a-40e7-bad1-4c162010e570 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:01:03.260175 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00619e4c-9a20-46f3-b3dc-f1839840e08c None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:01:03.302889 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:01:03.306337 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a402e0-2977-40a8-a9da-c61d08bb4b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:01:03.306989 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a402e0-2977-40a8-a9da-c61d08bb4b38 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:03.307320 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a402e0-2977-40a8-a9da-c61d08bb4b38 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:03.307600 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a402e0-2977-40a8-a9da-c61d08bb4b38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:03.326718 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a402e0-2977-40a8-a9da-c61d08bb4b38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:01:03.328513 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa7ce18-7e4a-40e7-bad1-4c162010e570 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:01:03.329167 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a402e0-2977-40a8-a9da-c61d08bb4b38 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:01:03.355532 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-36a402e0-2977-40a8-a9da-c61d08bb4b38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 20:01:03.355532 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:01:03.355532 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:01:03.355532 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:01:03.355532 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:01:03.355532 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:01:03.355532 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:01:03.355532 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:01:03.355532 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:01:03.355532 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:01:03.355532 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:01:03.355532 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:01:03.355532 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 20:01:03.355532 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 20:01:03.355532 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 20:01:03.355532 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 20:01:03.357473 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:01:03.357473 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:01:03.357473 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:01:03.357473 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:01:03.357473 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:01:03.357473 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:01:03.357473 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:01:03.357473 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 20:01:03.357473 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:01:03.357473 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:01:03.357473 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:01:03.357473 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:01:03.357473 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:01:03.357473 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:01:03.357473 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:01:03.357473 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:01:03.357473 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 20:01:03.359976 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 20:01:03.359976 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 20:01:03.359976 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:01:03.359976 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 525/1078] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 20:01:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 20:01:03.365101 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a402e0-2977-40a8-a9da-c61d08bb4b38 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:01:03.383365 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae543901-4c9b-44da-b911-308b461d7aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:01:03.384371 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae543901-4c9b-44da-b911-308b461d7aac None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:03.384699 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae543901-4c9b-44da-b911-308b461d7aac None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:03.385049 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae543901-4c9b-44da-b911-308b461d7aac None admin] PATH_INFO: `/v3/projects/da8eb4b247e14d629bb0e825fe00b39d` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:03.387519 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae543901-4c9b-44da-b911-308b461d7aac None admin] RBAC: Authorizing `identity:get_project(project_id=da8eb4b247e14d629bb0e825fe00b39d)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:01:03.390274 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae543901-4c9b-44da-b911-308b461d7aac None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:01:03.393252 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 554/1079] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 6 20:01:03 2022] GET /identity/v3/projects/da8eb4b247e14d629bb0e825fe00b39d => generated 311 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:01:03.400392 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae543901-4c9b-44da-b911-308b461d7aac None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:01:03.411610 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:01:03.534030 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a402e0-2977-40a8-a9da-c61d08bb4b38 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:01:03.548514 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae543901-4c9b-44da-b911-308b461d7aac None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:01:03.591534 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0c1ce6-6e7a-4852-b3af-2b79cce3da63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:01:03.592243 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0c1ce6-6e7a-4852-b3af-2b79cce3da63 None admin] REQUEST_METHOD: `PUT` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:03.592742 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0c1ce6-6e7a-4852-b3af-2b79cce3da63 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:03.593050 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0c1ce6-6e7a-4852-b3af-2b79cce3da63 None admin] PATH_INFO: `/v3/projects/da8eb4b247e14d629bb0e825fe00b39d/users/cd152c34d6ec447fb7ba9f421f6bffc7/roles/e0ab41d8ae22474688b5361bcc75e0f0` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:03.596034 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0c1ce6-6e7a-4852-b3af-2b79cce3da63 None admin] RBAC: Authorizing `identity:create_grant(project_id=da8eb4b247e14d629bb0e825fe00b39d, user_id=cd152c34d6ec447fb7ba9f421f6bffc7, role_id=e0ab41d8ae22474688b5361bcc75e0f0)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:01:03.598364 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0c1ce6-6e7a-4852-b3af-2b79cce3da63 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:01:03.601583 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ede2407-5047-4411-88a7-b62c9d622c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:01:03.602655 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ede2407-5047-4411-88a7-b62c9d622c9c None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:03.603146 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ede2407-5047-4411-88a7-b62c9d622c9c None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:03.603614 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ede2407-5047-4411-88a7-b62c9d622c9c None admin] PATH_INFO: `/v3/domains` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:03.604875 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ede2407-5047-4411-88a7-b62c9d622c9c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:01:03.605497 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ede2407-5047-4411-88a7-b62c9d622c9c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:01:03.609009 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ede2407-5047-4411-88a7-b62c9d622c9c None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:01:03.619263 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 555/1080] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Sep 6 20:01:03 2022] PUT /identity/v3/projects/da8eb4b247e14d629bb0e825fe00b39d/users/cd152c34d6ec447fb7ba9f421f6bffc7/roles/e0ab41d8ae22474688b5361bcc75e0f0 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 20:01:03.635151 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 526/1081] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 6 20:01:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 20:01:03.643313 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0c1ce6-6e7a-4852-b3af-2b79cce3da63 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:01:03.664786 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:01:03.816690 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0c1ce6-6e7a-4852-b3af-2b79cce3da63 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:01:03.884405 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b97765-4270-4912-94dc-db629cac7cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:01:03.885319 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b97765-4270-4912-94dc-db629cac7cf6 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:03.885800 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b97765-4270-4912-94dc-db629cac7cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:03.886340 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b97765-4270-4912-94dc-db629cac7cf6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:03.899742 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b97765-4270-4912-94dc-db629cac7cf6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:01:03.902777 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b97765-4270-4912-94dc-db629cac7cf6 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:01:03.925103 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-85b97765-4270-4912-94dc-db629cac7cf6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 20:01:03.925103 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:01:03.925103 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:01:03.925103 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:01:03.925103 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:01:03.925103 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:01:03.925103 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:01:03.925103 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:01:03.925103 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:01:03.925103 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:01:03.925103 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:01:03.925103 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:01:03.925103 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 20:01:03.925103 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 20:01:03.925103 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 20:01:03.926959 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 20:01:03.926959 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:01:03.926959 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:01:03.926959 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:01:03.926959 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:01:03.926959 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:01:03.926959 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:01:03.926959 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:01:03.926959 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 20:01:03.926959 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:01:03.926959 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:01:03.926959 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:01:03.926959 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:01:03.926959 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:01:03.926959 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:01:03.926959 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:01:03.926959 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:01:03.928865 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 20:01:03.928865 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 20:01:03.928865 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 20:01:03.928865 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 20:01:03.928865 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 20:01:03.928865 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 20:01:03.928865 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 20:01:03.928865 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 20:01:03.928865 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 556/1082] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 6 20:01:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 20:01:03.936842 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ede2407-5047-4411-88a7-b62c9d622c9c None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:01:04.059397 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:01:04.074017 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ede2407-5047-4411-88a7-b62c9d622c9c None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:01:04.119269 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056fa8bc-a8e5-4aaa-bdf9-4d555fad37fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:01:04.120156 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056fa8bc-a8e5-4aaa-bdf9-4d555fad37fe None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:04.120301 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056fa8bc-a8e5-4aaa-bdf9-4d555fad37fe None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:04.120591 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056fa8bc-a8e5-4aaa-bdf9-4d555fad37fe None admin] PATH_INFO: `/v3/projects` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:04.121404 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056fa8bc-a8e5-4aaa-bdf9-4d555fad37fe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:01:04.121777 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056fa8bc-a8e5-4aaa-bdf9-4d555fad37fe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:01:04.123317 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056fa8bc-a8e5-4aaa-bdf9-4d555fad37fe None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:01:04.140400 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 527/1083] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Sep 6 20:01:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:01:04.147089 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b97765-4270-4912-94dc-db629cac7cf6 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:01:04.245069 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:01:04.288428 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b97765-4270-4912-94dc-db629cac7cf6 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:01:04.337280 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7b9519-d2bc-4e4c-816f-0cde7fba21ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:01:04.338082 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7b9519-d2bc-4e4c-816f-0cde7fba21ad None admin] REQUEST_METHOD: `PUT` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:04.338427 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7b9519-d2bc-4e4c-816f-0cde7fba21ad None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:04.338808 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7b9519-d2bc-4e4c-816f-0cde7fba21ad None admin] PATH_INFO: `/v3/projects/6aef43493bb1420a9e122313d8db7d5a/users/5776cbf4f6714b2083c3ebdd6d98cc87/roles/960d5e4b29394e7aae95d5a428dedfc0` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:04.342022 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7b9519-d2bc-4e4c-816f-0cde7fba21ad None admin] RBAC: Authorizing `identity:create_grant(project_id=6aef43493bb1420a9e122313d8db7d5a, user_id=5776cbf4f6714b2083c3ebdd6d98cc87, role_id=960d5e4b29394e7aae95d5a428dedfc0)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:01:04.343582 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7b9519-d2bc-4e4c-816f-0cde7fba21ad None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:01:04.362008 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 557/1084] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Sep 6 20:01:04 2022] PUT /identity/v3/projects/6aef43493bb1420a9e122313d8db7d5a/users/5776cbf4f6714b2083c3ebdd6d98cc87/roles/960d5e4b29394e7aae95d5a428dedfc0 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 20:01:06.222669 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c00cc3-a3ff-4a60-be2a-7846e8cb1b8f None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:06.223376 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c00cc3-a3ff-4a60-be2a-7846e8cb1b8f None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:06.223590 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c00cc3-a3ff-4a60-be2a-7846e8cb1b8f None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:06.226291 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 528/1085] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:01:06 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:01:06.236814 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65e554b-ce80-4e08-98dd-e2a77851d438 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:06.237492 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65e554b-ce80-4e08-98dd-e2a77851d438 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:06.237992 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65e554b-ce80-4e08-98dd-e2a77851d438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:06.271571 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-b65e554b-ce80-4e08-98dd-e2a77851d438 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:01:06.273833 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-b65e554b-ce80-4e08-98dd-e2a77851d438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:01:06.368637 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 558/1086] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:01:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 20:01:06.685789 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c00cc3-a3ff-4a60-be2a-7846e8cb1b8f None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:01:06.869311 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c00cc3-a3ff-4a60-be2a-7846e8cb1b8f None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:01:06.919472 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf1abf5-a497-4513-9ebb-0ef0e3e3ff5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:01:06.920174 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf1abf5-a497-4513-9ebb-0ef0e3e3ff5c None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:06.920472 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf1abf5-a497-4513-9ebb-0ef0e3e3ff5c None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:06.920730 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf1abf5-a497-4513-9ebb-0ef0e3e3ff5c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:06.935495 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf1abf5-a497-4513-9ebb-0ef0e3e3ff5c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 20:01:06.941018 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-bbf1abf5-a497-4513-9ebb-0ef0e3e3ff5c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 20:01:06.941018 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:01:06.941018 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:01:06.941018 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:01:06.941018 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:01:06.941018 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:01:06.941018 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:01:06.941018 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:01:06.941018 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:01:06.941018 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:01:06.941018 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:01:06.941018 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:01:06.941018 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 20:01:06.941018 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 20:01:06.941018 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 20:01:06.941018 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 20:01:06.942930 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 20:01:06.942930 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 20:01:06.942930 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 20:01:06.942930 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 20:01:06.942930 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:01:06.942930 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:01:06.942930 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:01:06.942930 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:01:06.942930 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:01:06.942930 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:01:06.942930 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:01:06.942930 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 20:01:06.942930 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:01:06.942930 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:01:06.942930 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:01:06.942930 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:01:06.944598 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:01:06.944598 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:01:06.944598 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:01:06.944598 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:01:06.944598 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 20:01:06.944598 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 20:01:06.944598 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 20:01:06.944598 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 20:01:06.944598 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 20:01:06.944598 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 20:01:06.944598 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 20:01:06.944598 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:01:06.944598 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 529/1087] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Sep 6 20:01:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:01:06.946076 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bc2839-9d60-4aaa-8eda-aa2ed329b065 None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:06.946076 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bc2839-9d60-4aaa-8eda-aa2ed329b065 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:06.946076 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bc2839-9d60-4aaa-8eda-aa2ed329b065 None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:06.946076 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 559/1088] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:01:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:01:06.950740 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf1abf5-a497-4513-9ebb-0ef0e3e3ff5c None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:01:06.952512 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d890f36-8e1f-4d4f-bdbd-9f6c79c3d3f7 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:06.952970 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d890f36-8e1f-4d4f-bdbd-9f6c79c3d3f7 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:06.953644 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d890f36-8e1f-4d4f-bdbd-9f6c79c3d3f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:06.970450 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:01:06.975237 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-2d890f36-8e1f-4d4f-bdbd-9f6c79c3d3f7 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:01:06.976574 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-2d890f36-8e1f-4d4f-bdbd-9f6c79c3d3f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:01:07.070605 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:01:07.094192 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 560/1089] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:01:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 20:01:07.122053 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf1abf5-a497-4513-9ebb-0ef0e3e3ff5c None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:01:07.164437 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4700caa1-f961-4990-8a3e-006b658feb75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:01:07.165057 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4700caa1-f961-4990-8a3e-006b658feb75 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:07.165338 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4700caa1-f961-4990-8a3e-006b658feb75 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:07.165635 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4700caa1-f961-4990-8a3e-006b658feb75 None admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:07.166337 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4700caa1-f961-4990-8a3e-006b658feb75 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:01:07.166639 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4700caa1-f961-4990-8a3e-006b658feb75 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:01:07.168609 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4700caa1-f961-4990-8a3e-006b658feb75 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:01:07.179486 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 530/1090] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 6 20:01:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 20:01:07.187115 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d890f36-8e1f-4d4f-bdbd-9f6c79c3d3f7 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:01:07.315641 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d890f36-8e1f-4d4f-bdbd-9f6c79c3d3f7 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:01:07.357110 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67588f5-d49d-47fa-ab74-ee9230bf6a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:01:07.357838 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67588f5-d49d-47fa-ab74-ee9230bf6a96 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:07.358126 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67588f5-d49d-47fa-ab74-ee9230bf6a96 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:07.358405 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67588f5-d49d-47fa-ab74-ee9230bf6a96 None admin] PATH_INFO: `/v3/users/cd152c34d6ec447fb7ba9f421f6bffc7` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:07.359683 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67588f5-d49d-47fa-ab74-ee9230bf6a96 None admin] RBAC: Authorizing `identity:get_user(user_id=cd152c34d6ec447fb7ba9f421f6bffc7)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:01:07.361331 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67588f5-d49d-47fa-ab74-ee9230bf6a96 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:01:07.363124 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 561/1091] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 6 20:01:07 2022] GET /identity/v3/users/cd152c34d6ec447fb7ba9f421f6bffc7 => generated 285 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:01:07.369907 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4700caa1-f961-4990-8a3e-006b658feb75 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:01:07.458902 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:01:07.498305 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4700caa1-f961-4990-8a3e-006b658feb75 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:01:07.542345 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a039d270-efc7-4328-83dd-037cbaa2aa6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:01:07.542839 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a039d270-efc7-4328-83dd-037cbaa2aa6d None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:07.543116 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a039d270-efc7-4328-83dd-037cbaa2aa6d None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:07.543392 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a039d270-efc7-4328-83dd-037cbaa2aa6d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:07.621144 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67588f5-d49d-47fa-ab74-ee9230bf6a96 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:01:07.636164 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a039d270-efc7-4328-83dd-037cbaa2aa6d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:01:07.637958 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a039d270-efc7-4328-83dd-037cbaa2aa6d None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:01:07.726214 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:01:07.731669 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:01:07.783556 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 531/1092] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 6 20:01:07 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 06 20:01:07.790984 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a039d270-efc7-4328-83dd-037cbaa2aa6d None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:01:07.797023 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67588f5-d49d-47fa-ab74-ee9230bf6a96 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:01:07.856693 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f79dbd7-eccb-46dc-bb5a-a00287ee6a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:01:07.857389 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f79dbd7-eccb-46dc-bb5a-a00287ee6a16 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:07.857762 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f79dbd7-eccb-46dc-bb5a-a00287ee6a16 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:07.858074 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f79dbd7-eccb-46dc-bb5a-a00287ee6a16 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:07.876856 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f79dbd7-eccb-46dc-bb5a-a00287ee6a16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 20:01:07.887531 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-6f79dbd7-eccb-46dc-bb5a-a00287ee6a16 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 06 20:01:07.887531 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:01:07.887531 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:01:07.887531 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:01:07.887531 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:01:07.887531 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:01:07.887531 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:01:07.887531 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:01:07.887531 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:01:07.887531 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:01:07.887531 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:01:07.887531 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:01:07.887531 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 20:01:07.887531 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 20:01:07.887531 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 20:01:07.887531 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 20:01:07.889939 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 20:01:07.889939 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 20:01:07.889939 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 20:01:07.889939 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 20:01:07.889939 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:01:07.889939 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:01:07.889939 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:01:07.889939 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:01:07.889939 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:01:07.889939 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:01:07.889939 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:01:07.889939 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 20:01:07.889939 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:01:07.889939 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:01:07.889939 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:01:07.889939 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:01:07.891631 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:01:07.891631 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:01:07.891631 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:01:07.891631 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:01:07.891631 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 20:01:07.891631 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 20:01:07.891631 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 20:01:07.891631 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 20:01:07.891631 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 20:01:07.891631 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 20:01:07.891631 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 06 20:01:07.891631 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 20:01:07.891631 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 562/1093] 10.222.0.20 () {56 vars in 1120 bytes} [Tue Sep 6 20:01:07 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:01:07.896782 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f79dbd7-eccb-46dc-bb5a-a00287ee6a16 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:01:07.951060 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a039d270-efc7-4328-83dd-037cbaa2aa6d None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:01:08.000691 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcc86c2-5b1d-420c-a722-833f030fa08b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:01:08.001575 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcc86c2-5b1d-420c-a722-833f030fa08b None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:08.002067 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcc86c2-5b1d-420c-a722-833f030fa08b None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:08.002519 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcc86c2-5b1d-420c-a722-833f030fa08b None admin] PATH_INFO: `/v3/projects/da8eb4b247e14d629bb0e825fe00b39d` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:08.004626 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcc86c2-5b1d-420c-a722-833f030fa08b None admin] RBAC: Authorizing `identity:get_project(project_id=da8eb4b247e14d629bb0e825fe00b39d)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:01:08.006808 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcc86c2-5b1d-420c-a722-833f030fa08b None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:01:08.010061 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 532/1094] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 6 20:01:07 2022] GET /identity/v3/projects/da8eb4b247e14d629bb0e825fe00b39d => generated 311 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 20:01:08.020564 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcc86c2-5b1d-420c-a722-833f030fa08b None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:01:08.051651 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f79dbd7-eccb-46dc-bb5a-a00287ee6a16 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:01:08.074533 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:01:08.084007 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:01:08.102062 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277d18f5-b255-4d03-9867-e2f2a9e61a00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:01:08.102922 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277d18f5-b255-4d03-9867-e2f2a9e61a00 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:08.103244 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277d18f5-b255-4d03-9867-e2f2a9e61a00 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:08.103525 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277d18f5-b255-4d03-9867-e2f2a9e61a00 None admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:08.104386 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277d18f5-b255-4d03-9867-e2f2a9e61a00 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:01:08.104814 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277d18f5-b255-4d03-9867-e2f2a9e61a00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:01:08.106416 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277d18f5-b255-4d03-9867-e2f2a9e61a00 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:01:08.118828 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 563/1095] 10.222.0.20 () {56 vars in 1115 bytes} [Tue Sep 6 20:01:07 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 20:01:08.127929 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277d18f5-b255-4d03-9867-e2f2a9e61a00 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:01:08.186270 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcc86c2-5b1d-420c-a722-833f030fa08b None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:01:08.230824 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da45174f-9cf8-4185-bf45-197fa8969c18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:01:08.231695 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da45174f-9cf8-4185-bf45-197fa8969c18 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:08.232064 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da45174f-9cf8-4185-bf45-197fa8969c18 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:08.232566 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da45174f-9cf8-4185-bf45-197fa8969c18 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:08.233326 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da45174f-9cf8-4185-bf45-197fa8969c18 None admin] RBAC: Adding query filter params (role.id=e0ab41d8ae22474688b5361bcc75e0f0, scope.project.id=da8eb4b247e14d629bb0e825fe00b39d, user.id=cd152c34d6ec447fb7ba9f421f6bffc7) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:01:08.233682 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da45174f-9cf8-4185-bf45-197fa8969c18 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:01:08.235425 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da45174f-9cf8-4185-bf45-197fa8969c18 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:01:08.242829 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 533/1096] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Sep 6 20:01:08 2022] GET /identity/v3/role_assignments?user.id=cd152c34d6ec447fb7ba9f421f6bffc7&scope.project.id=da8eb4b247e14d629bb0e825fe00b39d&role.id=e0ab41d8ae22474688b5361bcc75e0f0 => generated 608 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:01:08.295169 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277d18f5-b255-4d03-9867-e2f2a9e61a00 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:01:08.341131 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e37abf7-ce7a-42ec-8f22-2eeaab22a7f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:01:08.341994 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e37abf7-ce7a-42ec-8f22-2eeaab22a7f2 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:08.342132 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e37abf7-ce7a-42ec-8f22-2eeaab22a7f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:08.342405 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e37abf7-ce7a-42ec-8f22-2eeaab22a7f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:08.357998 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e37abf7-ce7a-42ec-8f22-2eeaab22a7f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:01:08.359623 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e37abf7-ce7a-42ec-8f22-2eeaab22a7f2 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:01:08.378576 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-2e37abf7-ce7a-42ec-8f22-2eeaab22a7f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 20:01:08.378576 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:01:08.378576 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:01:08.378576 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:01:08.378576 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:01:08.378576 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:01:08.378576 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:01:08.378576 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:01:08.378576 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:01:08.378576 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:01:08.378576 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:01:08.378576 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:01:08.378576 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 20:01:08.378576 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 20:01:08.378576 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 20:01:08.378576 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 20:01:08.380851 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:01:08.380851 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:01:08.380851 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:01:08.380851 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:01:08.380851 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:01:08.380851 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:01:08.380851 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:01:08.380851 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 20:01:08.380851 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:01:08.380851 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:01:08.380851 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:01:08.380851 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:01:08.380851 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:01:08.380851 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:01:08.380851 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:01:08.380851 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:01:08.380851 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 20:01:08.382007 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 20:01:08.382007 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 20:01:08.382007 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 20:01:08.382007 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 564/1097] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 20:01:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:01:08.390545 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da45174f-9cf8-4185-bf45-197fa8969c18 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:01:08.554125 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da45174f-9cf8-4185-bf45-197fa8969c18 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:01:08.559234 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:01:08.609142 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c894d6-445b-4db9-af30-4ac1717d3911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:01:08.609978 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c894d6-445b-4db9-af30-4ac1717d3911 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:08.610441 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c894d6-445b-4db9-af30-4ac1717d3911 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:08.610788 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c894d6-445b-4db9-af30-4ac1717d3911 None admin] PATH_INFO: `/v3/domains` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:08.611562 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c894d6-445b-4db9-af30-4ac1717d3911 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:01:08.611933 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c894d6-445b-4db9-af30-4ac1717d3911 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:01:08.613722 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c894d6-445b-4db9-af30-4ac1717d3911 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:01:08.634180 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 534/1098] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 6 20:01:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:01:08.645267 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e37abf7-ce7a-42ec-8f22-2eeaab22a7f2 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:01:08.693557 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:01:08.803441 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e37abf7-ce7a-42ec-8f22-2eeaab22a7f2 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:01:08.866228 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fa8628-f8ce-44aa-a8b7-b42f032d3560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:01:08.867336 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fa8628-f8ce-44aa-a8b7-b42f032d3560 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:08.867889 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fa8628-f8ce-44aa-a8b7-b42f032d3560 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:08.868402 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fa8628-f8ce-44aa-a8b7-b42f032d3560 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:08.882723 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fa8628-f8ce-44aa-a8b7-b42f032d3560 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:01:08.885297 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fa8628-f8ce-44aa-a8b7-b42f032d3560 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:01:08.910575 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-86fa8628-f8ce-44aa-a8b7-b42f032d3560 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 06 20:01:08.910575 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:01:08.910575 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:01:08.910575 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:01:08.910575 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:01:08.910575 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:01:08.910575 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:01:08.910575 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:01:08.910575 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:01:08.910575 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:01:08.910575 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:01:08.910575 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:01:08.910575 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 20:01:08.910575 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 20:01:08.910575 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 20:01:08.910575 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 20:01:08.911998 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:01:08.911998 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:01:08.911998 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 20:01:08.911998 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 20:01:08.911998 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 20:01:08.911998 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 20:01:08.911998 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:01:08.911998 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:01:08.911998 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:01:08.911998 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:01:08.911998 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:01:08.911998 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 20:01:08.911998 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:01:08.911998 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:01:08.911998 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:01:08.911998 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:01:08.911998 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:01:08.911998 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:01:08.912980 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:01:08.912980 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:01:08.912980 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 20:01:08.912980 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 20:01:08.912980 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 20:01:08.912980 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 20:01:08.912980 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 20:01:08.912980 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 20:01:08.912980 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 06 20:01:08.912980 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 20:01:08.912980 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 565/1099] 10.222.0.20 () {56 vars in 1093 bytes} [Tue Sep 6 20:01:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:01:08.918292 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c894d6-445b-4db9-af30-4ac1717d3911 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:01:09.132165 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c894d6-445b-4db9-af30-4ac1717d3911 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:01:09.200885 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0df749-f64e-4bc3-89ce-e41ab50bc01a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:01:09.202226 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0df749-f64e-4bc3-89ce-e41ab50bc01a None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:09.202448 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0df749-f64e-4bc3-89ce-e41ab50bc01a None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:09.202918 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0df749-f64e-4bc3-89ce-e41ab50bc01a None admin] PATH_INFO: `/v3/users` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:09.204347 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0df749-f64e-4bc3-89ce-e41ab50bc01a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:01:09.204890 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0df749-f64e-4bc3-89ce-e41ab50bc01a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:01:09.207663 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0df749-f64e-4bc3-89ce-e41ab50bc01a None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:01:09.243789 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 535/1100] 10.222.0.20 () {56 vars in 1133 bytes} [Tue Sep 6 20:01:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:01:09.251672 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fa8628-f8ce-44aa-a8b7-b42f032d3560 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:01:09.384674 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:01:09.397811 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fa8628-f8ce-44aa-a8b7-b42f032d3560 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:01:09.443259 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f168f1-5650-4ab3-8dc9-857883ace755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:01:09.444308 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f168f1-5650-4ab3-8dc9-857883ace755 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:09.444851 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f168f1-5650-4ab3-8dc9-857883ace755 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:09.445369 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f168f1-5650-4ab3-8dc9-857883ace755 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:09.539979 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f168f1-5650-4ab3-8dc9-857883ace755 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:01:09.541993 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f168f1-5650-4ab3-8dc9-857883ace755 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:01:09.668672 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:01:09.681737 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 566/1101] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 6 20:01:09 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 20:01:09.689939 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0df749-f64e-4bc3-89ce-e41ab50bc01a None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:01:09.725374 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:01:09.828218 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0df749-f64e-4bc3-89ce-e41ab50bc01a None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:01:09.871797 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b3d6dd-f0c6-4035-a747-b6d660b2f1f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:01:09.872569 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b3d6dd-f0c6-4035-a747-b6d660b2f1f8 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:09.872935 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b3d6dd-f0c6-4035-a747-b6d660b2f1f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:09.873309 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b3d6dd-f0c6-4035-a747-b6d660b2f1f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:09.885303 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b3d6dd-f0c6-4035-a747-b6d660b2f1f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:01:09.887095 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b3d6dd-f0c6-4035-a747-b6d660b2f1f8 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:01:09.905530 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-63b3d6dd-f0c6-4035-a747-b6d660b2f1f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 20:01:09.905530 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:01:09.905530 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:01:09.905530 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:01:09.905530 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:01:09.905530 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:01:09.905530 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:01:09.905530 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:01:09.905530 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:01:09.905530 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:01:09.905530 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:01:09.905530 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:01:09.905530 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 20:01:09.905530 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 20:01:09.905530 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 20:01:09.905530 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 20:01:09.906933 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:01:09.906933 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:01:09.906933 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:01:09.906933 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:01:09.906933 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:01:09.906933 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:01:09.906933 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:01:09.906933 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 20:01:09.906933 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:01:09.906933 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:01:09.906933 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:01:09.906933 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:01:09.906933 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:01:09.906933 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:01:09.906933 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:01:09.906933 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:01:09.906933 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 20:01:09.908236 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 20:01:09.908236 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 20:01:09.908236 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:01:09.908236 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 536/1102] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 20:01:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:01:09.914396 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f168f1-5650-4ab3-8dc9-857883ace755 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:01:10.044910 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f168f1-5650-4ab3-8dc9-857883ace755 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:01:10.090077 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d829a0d2-783a-4314-b1a6-4f60c48b016d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:01:10.090822 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d829a0d2-783a-4314-b1a6-4f60c48b016d None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:10.091127 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d829a0d2-783a-4314-b1a6-4f60c48b016d None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:10.091417 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d829a0d2-783a-4314-b1a6-4f60c48b016d None admin] PATH_INFO: `/v3/domains` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:10.092188 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d829a0d2-783a-4314-b1a6-4f60c48b016d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:01:10.092554 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d829a0d2-783a-4314-b1a6-4f60c48b016d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:01:10.093967 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d829a0d2-783a-4314-b1a6-4f60c48b016d None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:01:10.109690 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 567/1103] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 6 20:01:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:01:10.116794 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b3d6dd-f0c6-4035-a747-b6d660b2f1f8 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:01:10.225132 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:01:10.264283 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b3d6dd-f0c6-4035-a747-b6d660b2f1f8 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:01:10.309507 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6985cba8-b4b8-4ed4-9663-4028c75eef4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:01:10.310111 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6985cba8-b4b8-4ed4-9663-4028c75eef4a None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:10.310390 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6985cba8-b4b8-4ed4-9663-4028c75eef4a None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:10.310696 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6985cba8-b4b8-4ed4-9663-4028c75eef4a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:10.320044 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6985cba8-b4b8-4ed4-9663-4028c75eef4a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:01:10.322636 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6985cba8-b4b8-4ed4-9663-4028c75eef4a None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:01:10.349272 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-6985cba8-b4b8-4ed4-9663-4028c75eef4a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 20:01:10.349272 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:01:10.349272 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:01:10.349272 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:01:10.349272 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:01:10.349272 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:01:10.349272 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:01:10.349272 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:01:10.349272 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:01:10.349272 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:01:10.349272 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:01:10.349272 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:01:10.349272 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 20:01:10.349272 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 20:01:10.349272 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 20:01:10.351143 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 20:01:10.351143 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:01:10.351143 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:01:10.351143 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:01:10.351143 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:01:10.351143 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:01:10.351143 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:01:10.351143 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:01:10.351143 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 20:01:10.351143 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:01:10.351143 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:01:10.351143 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:01:10.351143 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:01:10.351143 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:01:10.351143 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:01:10.351143 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:01:10.351143 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:01:10.353484 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 20:01:10.353484 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 20:01:10.353484 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 20:01:10.353484 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 20:01:10.353484 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 20:01:10.353484 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 20:01:10.353484 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 20:01:10.353484 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:01:10.353484 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 537/1104] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 6 20:01:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 20:01:10.362029 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d829a0d2-783a-4314-b1a6-4f60c48b016d None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:01:10.441395 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:01:10.506458 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d829a0d2-783a-4314-b1a6-4f60c48b016d None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:01:10.551088 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369fa100-c64f-454a-99f5-93f667d884dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:01:10.552256 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369fa100-c64f-454a-99f5-93f667d884dc None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:10.552652 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369fa100-c64f-454a-99f5-93f667d884dc None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:10.553057 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369fa100-c64f-454a-99f5-93f667d884dc None admin] PATH_INFO: `/v3/projects` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:10.553962 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369fa100-c64f-454a-99f5-93f667d884dc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:01:10.554433 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369fa100-c64f-454a-99f5-93f667d884dc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:01:10.556288 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369fa100-c64f-454a-99f5-93f667d884dc None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:01:10.575352 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 568/1105] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Sep 6 20:01:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:01:10.582318 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6985cba8-b4b8-4ed4-9663-4028c75eef4a None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:01:10.726268 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6985cba8-b4b8-4ed4-9663-4028c75eef4a None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:01:10.772312 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dcffad-9e0a-45c8-a631-c636952d03fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:01:10.773040 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dcffad-9e0a-45c8-a631-c636952d03fe None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:10.773348 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dcffad-9e0a-45c8-a631-c636952d03fe None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:10.773733 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dcffad-9e0a-45c8-a631-c636952d03fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:10.774590 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dcffad-9e0a-45c8-a631-c636952d03fe None admin] RBAC: Adding query filter params (role.id=960d5e4b29394e7aae95d5a428dedfc0, scope.project.id=6aef43493bb1420a9e122313d8db7d5a, user.id=5776cbf4f6714b2083c3ebdd6d98cc87) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:01:10.775033 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dcffad-9e0a-45c8-a631-c636952d03fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:01:10.776719 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dcffad-9e0a-45c8-a631-c636952d03fe None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:01:10.779265 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:01:10.789324 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 538/1106] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Sep 6 20:01:10 2022] GET /identity/v3/role_assignments?user.id=5776cbf4f6714b2083c3ebdd6d98cc87&scope.project.id=6aef43493bb1420a9e122313d8db7d5a&role.id=960d5e4b29394e7aae95d5a428dedfc0 => generated 608 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 20:01:10.933004 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b060e67d-53b9-42ba-bbf3-7f18b8be0ebb None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:10.933511 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b060e67d-53b9-42ba-bbf3-7f18b8be0ebb None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:10.934047 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b060e67d-53b9-42ba-bbf3-7f18b8be0ebb None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:10.936643 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 569/1107] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:01:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:01:10.946548 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0d608b-c148-4685-8732-bcb44b08a51e None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:10.947052 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0d608b-c148-4685-8732-bcb44b08a51e None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:10.947658 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0d608b-c148-4685-8732-bcb44b08a51e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:10.980990 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-6e0d608b-c148-4685-8732-bcb44b08a51e None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:01:10.983317 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-6e0d608b-c148-4685-8732-bcb44b08a51e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:01:11.100880 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 539/1108] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:01:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 20:01:11.391654 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b060e67d-53b9-42ba-bbf3-7f18b8be0ebb None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:01:11.529005 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b060e67d-53b9-42ba-bbf3-7f18b8be0ebb None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:01:11.551536 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:01:11.591928 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d061a0-079d-4d6d-bb29-fcd6af498c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:01:11.593063 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d061a0-079d-4d6d-bb29-fcd6af498c86 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:11.593538 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d061a0-079d-4d6d-bb29-fcd6af498c86 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:11.594079 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d061a0-079d-4d6d-bb29-fcd6af498c86 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:11.596259 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d061a0-079d-4d6d-bb29-fcd6af498c86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:01:11.599140 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d061a0-079d-4d6d-bb29-fcd6af498c86 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:01:11.602247 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 570/1109] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 20:01:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:01:11.609748 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0d608b-c148-4685-8732-bcb44b08a51e None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:01:11.758952 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0d608b-c148-4685-8732-bcb44b08a51e None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:01:11.796322 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:01:11.809688 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f94a19-1281-4f68-ad56-177f4164b905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:01:11.810342 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f94a19-1281-4f68-ad56-177f4164b905 None admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:11.810626 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f94a19-1281-4f68-ad56-177f4164b905 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:11.810900 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f94a19-1281-4f68-ad56-177f4164b905 None admin] PATH_INFO: `/v3/users` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:11.811613 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f94a19-1281-4f68-ad56-177f4164b905 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:01:11.813100 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f94a19-1281-4f68-ad56-177f4164b905 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:01:11.847621 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 540/1110] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Sep 6 20:01:11 2022] POST /identity/v3/users => generated 286 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 20:01:14.309451 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfd7bfd-e9f9-4da6-9ff3-8576c34548f9 None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:14.310089 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfd7bfd-e9f9-4da6-9ff3-8576c34548f9 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:14.310275 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfd7bfd-e9f9-4da6-9ff3-8576c34548f9 None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:14.311617 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 571/1111] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:01:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:01:14.318448 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a385415-a255-4cda-b9fd-468b2e2ae1fa None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:14.319236 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a385415-a255-4cda-b9fd-468b2e2ae1fa None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:14.319390 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a385415-a255-4cda-b9fd-468b2e2ae1fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:14.342062 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-7a385415-a255-4cda-b9fd-468b2e2ae1fa None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:01:14.343514 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-7a385415-a255-4cda-b9fd-468b2e2ae1fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:01:14.428500 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 541/1112] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:01:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 20:01:14.706822 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfd7bfd-e9f9-4da6-9ff3-8576c34548f9 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:01:14.841909 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfd7bfd-e9f9-4da6-9ff3-8576c34548f9 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:01:14.898811 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4851e6-6440-4b79-a047-92e1c189325e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:01:14.899498 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4851e6-6440-4b79-a047-92e1c189325e None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:14.899876 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4851e6-6440-4b79-a047-92e1c189325e None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:14.900124 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4851e6-6440-4b79-a047-92e1c189325e None admin] PATH_INFO: `/v3/roles/b02170460f054c259e825ab6e35f8537` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:14.904164 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4851e6-6440-4b79-a047-92e1c189325e None admin] RBAC: Authorizing `identity:get_role(role_id=b02170460f054c259e825ab6e35f8537)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:01:14.906925 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4851e6-6440-4b79-a047-92e1c189325e None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:01:14.909564 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 572/1113] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 6 20:01:14 2022] GET /identity/v3/roles/b02170460f054c259e825ab6e35f8537 => generated 223 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 20:01:14.919546 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a385415-a255-4cda-b9fd-468b2e2ae1fa None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:01:15.094550 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a385415-a255-4cda-b9fd-468b2e2ae1fa None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:01:15.123119 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:01:15.162523 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8be3ed-10cd-4617-8740-01e5c72eeb3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:01:15.163625 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8be3ed-10cd-4617-8740-01e5c72eeb3f None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:15.164126 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8be3ed-10cd-4617-8740-01e5c72eeb3f None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:15.164618 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8be3ed-10cd-4617-8740-01e5c72eeb3f None admin] PATH_INFO: `/v3/users/28758754d15147c7bde66c07dcd62908` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:15.195885 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8be3ed-10cd-4617-8740-01e5c72eeb3f None admin] RBAC: Authorizing `identity:get_user(user_id=28758754d15147c7bde66c07dcd62908)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:01:15.198819 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8be3ed-10cd-4617-8740-01e5c72eeb3f None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:01:15.202233 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 542/1114] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 6 20:01:14 2022] GET /identity/v3/users/28758754d15147c7bde66c07dcd62908 => generated 286 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:01:15.209381 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4851e6-6440-4b79-a047-92e1c189325e None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:01:15.274031 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:01:15.343710 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4851e6-6440-4b79-a047-92e1c189325e None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:01:15.385458 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b53e331-9200-491e-ac40-d6d368a48510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:01:15.386531 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b53e331-9200-491e-ac40-d6d368a48510 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:15.386919 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b53e331-9200-491e-ac40-d6d368a48510 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:15.387206 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b53e331-9200-491e-ac40-d6d368a48510 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:15.471284 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b53e331-9200-491e-ac40-d6d368a48510 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:01:15.473467 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b53e331-9200-491e-ac40-d6d368a48510 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:01:15.534140 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:01:15.596657 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 573/1115] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 6 20:01:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 06 20:01:15.603403 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8be3ed-10cd-4617-8740-01e5c72eeb3f None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:01:15.731535 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8be3ed-10cd-4617-8740-01e5c72eeb3f None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:01:15.777577 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8553b637-438d-430a-a2fc-9b3746ea1d04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:01:15.778345 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8553b637-438d-430a-a2fc-9b3746ea1d04 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:15.778628 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8553b637-438d-430a-a2fc-9b3746ea1d04 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:15.778902 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8553b637-438d-430a-a2fc-9b3746ea1d04 None admin] PATH_INFO: `/v3/projects/da8eb4b247e14d629bb0e825fe00b39d` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:15.780189 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8553b637-438d-430a-a2fc-9b3746ea1d04 None admin] RBAC: Authorizing `identity:get_project(project_id=da8eb4b247e14d629bb0e825fe00b39d)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:01:15.781690 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8553b637-438d-430a-a2fc-9b3746ea1d04 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:01:15.783460 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 543/1116] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 6 20:01:15 2022] GET /identity/v3/projects/da8eb4b247e14d629bb0e825fe00b39d => generated 311 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:01:15.790068 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b53e331-9200-491e-ac40-d6d368a48510 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:01:15.931654 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b53e331-9200-491e-ac40-d6d368a48510 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:01:15.991651 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fc084a-cff3-4d9a-b226-8afe83740321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:01:15.992231 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fc084a-cff3-4d9a-b226-8afe83740321 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:15.992556 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fc084a-cff3-4d9a-b226-8afe83740321 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:15.992877 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fc084a-cff3-4d9a-b226-8afe83740321 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:15.993719 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fc084a-cff3-4d9a-b226-8afe83740321 None admin] RBAC: Adding query filter params (role.id=b02170460f054c259e825ab6e35f8537, scope.project.id=da8eb4b247e14d629bb0e825fe00b39d, user.id=28758754d15147c7bde66c07dcd62908) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:01:15.994079 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fc084a-cff3-4d9a-b226-8afe83740321 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:01:15.996238 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fc084a-cff3-4d9a-b226-8afe83740321 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:01:16.002928 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 574/1117] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Sep 6 20:01:15 2022] GET /identity/v3/role_assignments?user.id=28758754d15147c7bde66c07dcd62908&scope.project.id=da8eb4b247e14d629bb0e825fe00b39d&role.id=b02170460f054c259e825ab6e35f8537 => generated 259 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 20:01:18.338337 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab742e80-732f-4138-ac45-acaa158bbe2f None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:18.339035 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab742e80-732f-4138-ac45-acaa158bbe2f None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:18.339035 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab742e80-732f-4138-ac45-acaa158bbe2f None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:18.340734 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 544/1118] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:01:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:01:18.347027 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a50a6ce-e55b-479e-ae1e-ff298c7bdd4e None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:18.347694 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a50a6ce-e55b-479e-ae1e-ff298c7bdd4e None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:18.347694 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a50a6ce-e55b-479e-ae1e-ff298c7bdd4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:18.368109 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-8a50a6ce-e55b-479e-ae1e-ff298c7bdd4e None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:01:18.369503 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-8a50a6ce-e55b-479e-ae1e-ff298c7bdd4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:01:18.387590 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:01:18.465306 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 575/1119] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:01:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 20:01:18.769829 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab742e80-732f-4138-ac45-acaa158bbe2f None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:01:18.827706 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:01:18.904578 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab742e80-732f-4138-ac45-acaa158bbe2f None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:01:18.962512 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce573d63-7ed6-4900-86ae-17497cff569a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:01:18.963178 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce573d63-7ed6-4900-86ae-17497cff569a None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:18.963460 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce573d63-7ed6-4900-86ae-17497cff569a None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:18.963740 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce573d63-7ed6-4900-86ae-17497cff569a None admin] PATH_INFO: `/v3/roles/b02170460f054c259e825ab6e35f8537` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:18.965419 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce573d63-7ed6-4900-86ae-17497cff569a None admin] RBAC: Authorizing `identity:get_role(role_id=b02170460f054c259e825ab6e35f8537)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:01:18.966986 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce573d63-7ed6-4900-86ae-17497cff569a None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:01:18.968413 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 545/1120] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 6 20:01:18 2022] GET /identity/v3/roles/b02170460f054c259e825ab6e35f8537 => generated 223 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:01:18.975037 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a50a6ce-e55b-479e-ae1e-ff298c7bdd4e None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:01:19.098111 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a50a6ce-e55b-479e-ae1e-ff298c7bdd4e None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:01:19.139961 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a83bae-8d9b-472c-85ba-de55e12a3f54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:01:19.140621 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a83bae-8d9b-472c-85ba-de55e12a3f54 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:19.140907 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a83bae-8d9b-472c-85ba-de55e12a3f54 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:19.141182 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a83bae-8d9b-472c-85ba-de55e12a3f54 None admin] PATH_INFO: `/v3/users/28758754d15147c7bde66c07dcd62908` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:19.142463 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a83bae-8d9b-472c-85ba-de55e12a3f54 None admin] RBAC: Authorizing `identity:get_user(user_id=28758754d15147c7bde66c07dcd62908)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:01:19.143975 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a83bae-8d9b-472c-85ba-de55e12a3f54 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:01:19.145708 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 576/1121] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 6 20:01:18 2022] GET /identity/v3/users/28758754d15147c7bde66c07dcd62908 => generated 286 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:01:19.160448 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce573d63-7ed6-4900-86ae-17497cff569a None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:01:19.281884 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce573d63-7ed6-4900-86ae-17497cff569a None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:01:19.294790 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:01:19.325688 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c436641c-dce9-4d09-bb1c-706efedbf08a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:01:19.326345 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c436641c-dce9-4d09-bb1c-706efedbf08a None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:19.326623 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c436641c-dce9-4d09-bb1c-706efedbf08a None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:19.326897 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c436641c-dce9-4d09-bb1c-706efedbf08a None admin] PATH_INFO: `/v3/projects/da8eb4b247e14d629bb0e825fe00b39d` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:19.328195 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c436641c-dce9-4d09-bb1c-706efedbf08a None admin] RBAC: Authorizing `identity:get_project(project_id=da8eb4b247e14d629bb0e825fe00b39d)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:01:19.329698 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c436641c-dce9-4d09-bb1c-706efedbf08a None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:01:19.331573 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 546/1122] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 6 20:01:19 2022] GET /identity/v3/projects/da8eb4b247e14d629bb0e825fe00b39d => generated 311 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 20:01:19.342881 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a83bae-8d9b-472c-85ba-de55e12a3f54 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:01:19.366841 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:01:19.539542 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a83bae-8d9b-472c-85ba-de55e12a3f54 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:01:19.607104 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edd382c-9345-4d32-8089-e0b6c701cafd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:01:19.608256 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edd382c-9345-4d32-8089-e0b6c701cafd None admin] REQUEST_METHOD: `PUT` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:19.608741 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edd382c-9345-4d32-8089-e0b6c701cafd None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:19.609218 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edd382c-9345-4d32-8089-e0b6c701cafd None admin] PATH_INFO: `/v3/projects/da8eb4b247e14d629bb0e825fe00b39d/users/28758754d15147c7bde66c07dcd62908/roles/b02170460f054c259e825ab6e35f8537` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:19.613213 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edd382c-9345-4d32-8089-e0b6c701cafd None admin] RBAC: Authorizing `identity:create_grant(project_id=da8eb4b247e14d629bb0e825fe00b39d, user_id=28758754d15147c7bde66c07dcd62908, role_id=b02170460f054c259e825ab6e35f8537)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:01:19.616398 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edd382c-9345-4d32-8089-e0b6c701cafd None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:01:19.641349 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 577/1123] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Sep 6 20:01:19 2022] PUT /identity/v3/projects/da8eb4b247e14d629bb0e825fe00b39d/users/28758754d15147c7bde66c07dcd62908/roles/b02170460f054c259e825ab6e35f8537 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:01:22.048003 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776abf8e-d55d-4459-88a4-a623ba6c7153 None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:22.048564 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776abf8e-d55d-4459-88a4-a623ba6c7153 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:22.048853 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776abf8e-d55d-4459-88a4-a623ba6c7153 None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:22.051072 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 547/1124] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:01:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:01:22.060293 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bc2aeb-5fcb-41f6-ad4e-3762a6ab8871 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:22.061155 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bc2aeb-5fcb-41f6-ad4e-3762a6ab8871 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:22.061155 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bc2aeb-5fcb-41f6-ad4e-3762a6ab8871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:22.084246 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-38bc2aeb-5fcb-41f6-ad4e-3762a6ab8871 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:01:22.085537 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-38bc2aeb-5fcb-41f6-ad4e-3762a6ab8871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:01:22.174068 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:01:22.181912 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 578/1125] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:01:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 20:01:22.482608 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776abf8e-d55d-4459-88a4-a623ba6c7153 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:01:22.678047 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776abf8e-d55d-4459-88a4-a623ba6c7153 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:01:22.729442 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afb485a-d120-481c-97d1-523ee837116d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:01:22.730798 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afb485a-d120-481c-97d1-523ee837116d None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:22.731169 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afb485a-d120-481c-97d1-523ee837116d None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:22.731534 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afb485a-d120-481c-97d1-523ee837116d None admin] PATH_INFO: `/v3/roles/b02170460f054c259e825ab6e35f8537` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:22.734008 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afb485a-d120-481c-97d1-523ee837116d None admin] RBAC: Authorizing `identity:get_role(role_id=b02170460f054c259e825ab6e35f8537)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:01:22.735440 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afb485a-d120-481c-97d1-523ee837116d None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:01:22.737144 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 548/1126] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 6 20:01:22 2022] GET /identity/v3/roles/b02170460f054c259e825ab6e35f8537 => generated 223 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:01:22.746587 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bc2aeb-5fcb-41f6-ad4e-3762a6ab8871 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:01:22.880333 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bc2aeb-5fcb-41f6-ad4e-3762a6ab8871 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:01:22.923560 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b1b535-4918-4432-b480-7e85b958e61d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:01:22.924215 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b1b535-4918-4432-b480-7e85b958e61d None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:22.924491 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b1b535-4918-4432-b480-7e85b958e61d None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:22.924769 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b1b535-4918-4432-b480-7e85b958e61d None admin] PATH_INFO: `/v3/users/28758754d15147c7bde66c07dcd62908` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:22.926014 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b1b535-4918-4432-b480-7e85b958e61d None admin] RBAC: Authorizing `identity:get_user(user_id=28758754d15147c7bde66c07dcd62908)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:01:22.927530 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b1b535-4918-4432-b480-7e85b958e61d None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:01:22.929305 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 579/1127] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 6 20:01:22 2022] GET /identity/v3/users/28758754d15147c7bde66c07dcd62908 => generated 286 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 20:01:22.935698 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afb485a-d120-481c-97d1-523ee837116d None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:01:22.995043 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:01:23.066170 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afb485a-d120-481c-97d1-523ee837116d None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:01:23.117093 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f866254e-aac8-49bb-b193-6167351413a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:01:23.117844 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f866254e-aac8-49bb-b193-6167351413a1 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:23.118130 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f866254e-aac8-49bb-b193-6167351413a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:23.118440 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f866254e-aac8-49bb-b193-6167351413a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:23.230610 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f866254e-aac8-49bb-b193-6167351413a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:01:23.232997 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f866254e-aac8-49bb-b193-6167351413a1 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:01:23.318435 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:01:23.429151 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 549/1128] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 6 20:01:22 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 20:01:23.437157 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b1b535-4918-4432-b480-7e85b958e61d None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:01:23.578592 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:01:23.619706 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b1b535-4918-4432-b480-7e85b958e61d None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:01:23.668297 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93b9ff2-9d09-4df6-9acb-427bac4cefc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:01:23.668987 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93b9ff2-9d09-4df6-9acb-427bac4cefc3 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:23.669317 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93b9ff2-9d09-4df6-9acb-427bac4cefc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:23.669593 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93b9ff2-9d09-4df6-9acb-427bac4cefc3 None admin] PATH_INFO: `/v3/projects/da8eb4b247e14d629bb0e825fe00b39d` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:23.670980 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93b9ff2-9d09-4df6-9acb-427bac4cefc3 None admin] RBAC: Authorizing `identity:get_project(project_id=da8eb4b247e14d629bb0e825fe00b39d)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:01:23.672476 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93b9ff2-9d09-4df6-9acb-427bac4cefc3 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:01:23.674263 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 580/1129] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 6 20:01:23 2022] GET /identity/v3/projects/da8eb4b247e14d629bb0e825fe00b39d => generated 311 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:01:23.680929 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f866254e-aac8-49bb-b193-6167351413a1 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:01:23.828302 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f866254e-aac8-49bb-b193-6167351413a1 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:01:23.873731 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48f9212-15dd-4927-9843-6b16d0d7abb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:01:23.874351 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48f9212-15dd-4927-9843-6b16d0d7abb2 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:23.874686 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48f9212-15dd-4927-9843-6b16d0d7abb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:23.874920 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48f9212-15dd-4927-9843-6b16d0d7abb2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:23.875693 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48f9212-15dd-4927-9843-6b16d0d7abb2 None admin] RBAC: Adding query filter params (role.id=b02170460f054c259e825ab6e35f8537, scope.project.id=da8eb4b247e14d629bb0e825fe00b39d, user.id=28758754d15147c7bde66c07dcd62908) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:01:23.875997 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48f9212-15dd-4927-9843-6b16d0d7abb2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:01:23.878499 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48f9212-15dd-4927-9843-6b16d0d7abb2 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:01:23.889236 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 550/1130] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Sep 6 20:01:23 2022] GET /identity/v3/role_assignments?user.id=28758754d15147c7bde66c07dcd62908&scope.project.id=da8eb4b247e14d629bb0e825fe00b39d&role.id=b02170460f054c259e825ab6e35f8537 => generated 608 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:01:26.254393 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b83c42-ac31-4450-bd17-be201b6896b8 None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:26.255014 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b83c42-ac31-4450-bd17-be201b6896b8 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:26.255014 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b83c42-ac31-4450-bd17-be201b6896b8 None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:26.256493 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 581/1131] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:01:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:01:26.262872 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a134896b-71db-4232-a0d9-b5a25a52adae None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:26.263694 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a134896b-71db-4232-a0d9-b5a25a52adae None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:26.263694 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a134896b-71db-4232-a0d9-b5a25a52adae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:26.283653 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-a134896b-71db-4232-a0d9-b5a25a52adae None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:01:26.285056 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-a134896b-71db-4232-a0d9-b5a25a52adae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:01:26.287500 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:01:26.366250 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 551/1132] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:01:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 20:01:26.649120 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b83c42-ac31-4450-bd17-be201b6896b8 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:01:26.770563 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b83c42-ac31-4450-bd17-be201b6896b8 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:01:26.814451 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4e784f-1d92-432d-8960-e64dd545e87c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:01:26.815021 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4e784f-1d92-432d-8960-e64dd545e87c None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:26.815297 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4e784f-1d92-432d-8960-e64dd545e87c None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:26.815570 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4e784f-1d92-432d-8960-e64dd545e87c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:26.816756 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4e784f-1d92-432d-8960-e64dd545e87c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:01:26.818303 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4e784f-1d92-432d-8960-e64dd545e87c None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:01:26.819967 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 582/1133] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 20:01:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:01:26.827082 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a134896b-71db-4232-a0d9-b5a25a52adae None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:01:26.971304 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a134896b-71db-4232-a0d9-b5a25a52adae None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:01:27.014884 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fee9b2b-7042-4d37-89a2-4698aa675537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:01:27.015515 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fee9b2b-7042-4d37-89a2-4698aa675537 None admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:27.015799 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fee9b2b-7042-4d37-89a2-4698aa675537 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:27.016071 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fee9b2b-7042-4d37-89a2-4698aa675537 None admin] PATH_INFO: `/v3/projects` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:27.016759 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fee9b2b-7042-4d37-89a2-4698aa675537 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:01:27.018396 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fee9b2b-7042-4d37-89a2-4698aa675537 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:01:27.029593 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:01:27.062160 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 552/1134] 10.222.0.20 () {60 vars in 1141 bytes} [Tue Sep 6 20:01:26 2022] POST /identity/v3/projects => generated 311 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:01:29.498668 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7660a9ca-eaf7-4b77-bbfa-0bf9b2f6521a None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:29.499533 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7660a9ca-eaf7-4b77-bbfa-0bf9b2f6521a None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:29.499533 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7660a9ca-eaf7-4b77-bbfa-0bf9b2f6521a None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:29.501858 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 583/1135] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:01:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 20:01:29.509107 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81280e02-aa77-4945-b47c-7cd6e4316a01 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:29.509707 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81280e02-aa77-4945-b47c-7cd6e4316a01 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:29.509828 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81280e02-aa77-4945-b47c-7cd6e4316a01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:29.529759 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-81280e02-aa77-4945-b47c-7cd6e4316a01 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:01:29.530988 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-81280e02-aa77-4945-b47c-7cd6e4316a01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:01:29.615520 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 553/1136] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:01:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 20:01:29.920753 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7660a9ca-eaf7-4b77-bbfa-0bf9b2f6521a None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:01:29.925082 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:01:30.098357 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7660a9ca-eaf7-4b77-bbfa-0bf9b2f6521a None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:01:30.169479 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91795a2f-96b4-4c1e-a5a9-de3049c1eb8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:01:30.170608 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91795a2f-96b4-4c1e-a5a9-de3049c1eb8c None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:30.171160 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91795a2f-96b4-4c1e-a5a9-de3049c1eb8c None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:30.171596 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91795a2f-96b4-4c1e-a5a9-de3049c1eb8c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:30.173783 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91795a2f-96b4-4c1e-a5a9-de3049c1eb8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:01:30.176572 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91795a2f-96b4-4c1e-a5a9-de3049c1eb8c None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:01:30.179752 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 584/1137] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 20:01:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 20:01:30.187154 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81280e02-aa77-4945-b47c-7cd6e4316a01 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:01:30.316378 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81280e02-aa77-4945-b47c-7cd6e4316a01 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:01:30.341882 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:01:30.359889 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0587dd1-2673-4294-ba96-b5d14d19229d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:01:30.360524 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0587dd1-2673-4294-ba96-b5d14d19229d None admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:30.360810 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0587dd1-2673-4294-ba96-b5d14d19229d None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:30.361090 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0587dd1-2673-4294-ba96-b5d14d19229d None admin] PATH_INFO: `/v3/users` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:30.361843 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0587dd1-2673-4294-ba96-b5d14d19229d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:01:30.363357 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0587dd1-2673-4294-ba96-b5d14d19229d None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:01:30.391043 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 554/1138] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Sep 6 20:01:30 2022] POST /identity/v3/users => generated 286 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 20:01:32.958663 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1ffa33-9cb6-48a7-add4-58d4ad5afc0d None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:32.959273 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1ffa33-9cb6-48a7-add4-58d4ad5afc0d None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:32.959273 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1ffa33-9cb6-48a7-add4-58d4ad5afc0d None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:32.960617 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 585/1139] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:01:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:01:32.968690 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2799e73c-09b5-416a-a36e-ac375f1200ee None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:32.969291 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2799e73c-09b5-416a-a36e-ac375f1200ee None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:32.969728 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2799e73c-09b5-416a-a36e-ac375f1200ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:33.001892 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-2799e73c-09b5-416a-a36e-ac375f1200ee None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:01:33.004031 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-2799e73c-09b5-416a-a36e-ac375f1200ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:01:33.133248 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 555/1140] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:01:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 20:01:33.425329 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1ffa33-9cb6-48a7-add4-58d4ad5afc0d None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:01:33.528308 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:01:33.567975 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1ffa33-9cb6-48a7-add4-58d4ad5afc0d None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:01:33.610565 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808f89d2-807a-4fcf-b812-03b81638c622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:01:33.611270 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808f89d2-807a-4fcf-b812-03b81638c622 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:33.611595 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808f89d2-807a-4fcf-b812-03b81638c622 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:33.611955 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808f89d2-807a-4fcf-b812-03b81638c622 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:33.625780 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808f89d2-807a-4fcf-b812-03b81638c622 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 20:01:33.631388 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-808f89d2-807a-4fcf-b812-03b81638c622 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 20:01:33.631388 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:01:33.631388 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:01:33.631388 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:01:33.631388 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:01:33.631388 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:01:33.631388 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:01:33.631388 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:01:33.631388 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:01:33.631388 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:01:33.631388 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:01:33.631388 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:01:33.631388 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 20:01:33.631388 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 20:01:33.631388 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 20:01:33.631388 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 20:01:33.633267 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 20:01:33.633267 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 20:01:33.633267 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 20:01:33.633267 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 20:01:33.633267 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:01:33.633267 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:01:33.633267 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:01:33.633267 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:01:33.633267 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:01:33.633267 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:01:33.633267 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:01:33.633267 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 20:01:33.633267 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:01:33.633267 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:01:33.633267 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:01:33.633267 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:01:33.635114 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:01:33.635114 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:01:33.635114 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:01:33.635114 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:01:33.635114 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 20:01:33.635114 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 20:01:33.635114 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 20:01:33.635114 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 20:01:33.635114 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 20:01:33.635114 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 20:01:33.635114 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 20:01:33.635114 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 20:01:33.635114 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 586/1141] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Sep 6 20:01:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 20:01:33.644090 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2799e73c-09b5-416a-a36e-ac375f1200ee None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:01:33.778923 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2799e73c-09b5-416a-a36e-ac375f1200ee None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:01:33.788306 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:01:33.826475 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2a5405-f422-4ce4-b07b-cfbaf7f8a88f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:01:33.827225 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2a5405-f422-4ce4-b07b-cfbaf7f8a88f None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:33.827726 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2a5405-f422-4ce4-b07b-cfbaf7f8a88f None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:33.828054 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2a5405-f422-4ce4-b07b-cfbaf7f8a88f None admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:33.828840 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2a5405-f422-4ce4-b07b-cfbaf7f8a88f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:01:33.829203 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2a5405-f422-4ce4-b07b-cfbaf7f8a88f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:01:33.830887 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2a5405-f422-4ce4-b07b-cfbaf7f8a88f None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:01:33.841837 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 556/1142] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 6 20:01:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 20:01:33.848733 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808f89d2-807a-4fcf-b812-03b81638c622 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:01:33.982549 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808f89d2-807a-4fcf-b812-03b81638c622 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:01:34.021810 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:01:34.028516 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee63fd57-0630-4755-945f-67f862267793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:01:34.029375 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee63fd57-0630-4755-945f-67f862267793 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:34.029711 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee63fd57-0630-4755-945f-67f862267793 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:34.030103 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee63fd57-0630-4755-945f-67f862267793 None admin] PATH_INFO: `/v3/users/06473bac23da480bb58714235678a732` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:34.048858 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee63fd57-0630-4755-945f-67f862267793 None admin] RBAC: Authorizing `identity:get_user(user_id=06473bac23da480bb58714235678a732)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:01:34.050619 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee63fd57-0630-4755-945f-67f862267793 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:01:34.052476 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 587/1143] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 6 20:01:33 2022] GET /identity/v3/users/06473bac23da480bb58714235678a732 => generated 286 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:01:34.062947 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2a5405-f422-4ce4-b07b-cfbaf7f8a88f None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:01:34.224239 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2a5405-f422-4ce4-b07b-cfbaf7f8a88f None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:01:34.267598 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2090f16d-0046-4ee2-823e-f5ee5ae6b2a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:01:34.268233 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2090f16d-0046-4ee2-823e-f5ee5ae6b2a1 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:34.268518 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2090f16d-0046-4ee2-823e-f5ee5ae6b2a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:34.268796 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2090f16d-0046-4ee2-823e-f5ee5ae6b2a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:34.313339 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:01:34.365447 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2090f16d-0046-4ee2-823e-f5ee5ae6b2a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:01:34.368293 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2090f16d-0046-4ee2-823e-f5ee5ae6b2a1 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:01:34.513219 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 557/1144] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 6 20:01:34 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 20:01:34.520025 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee63fd57-0630-4755-945f-67f862267793 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:01:34.638766 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee63fd57-0630-4755-945f-67f862267793 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:01:34.680606 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f754a1f-759f-47a2-ae27-470fa7f0dbe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:01:34.681272 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f754a1f-759f-47a2-ae27-470fa7f0dbe9 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:34.681554 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f754a1f-759f-47a2-ae27-470fa7f0dbe9 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:34.681864 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f754a1f-759f-47a2-ae27-470fa7f0dbe9 None admin] PATH_INFO: `/v3/projects/86d84d636c124b04a6a1a0a18417a17f` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:34.683111 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f754a1f-759f-47a2-ae27-470fa7f0dbe9 None admin] RBAC: Authorizing `identity:get_project(project_id=86d84d636c124b04a6a1a0a18417a17f)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:01:34.685168 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f754a1f-759f-47a2-ae27-470fa7f0dbe9 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:01:34.687005 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 588/1145] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 6 20:01:34 2022] GET /identity/v3/projects/86d84d636c124b04a6a1a0a18417a17f => generated 311 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:01:34.693863 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2090f16d-0046-4ee2-823e-f5ee5ae6b2a1 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:01:34.777977 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:01:34.831762 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2090f16d-0046-4ee2-823e-f5ee5ae6b2a1 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:01:34.871796 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d21eef-6927-4bad-a181-ef353a2553d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:01:34.872428 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d21eef-6927-4bad-a181-ef353a2553d4 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:34.872707 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d21eef-6927-4bad-a181-ef353a2553d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:34.872980 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d21eef-6927-4bad-a181-ef353a2553d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:34.873890 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d21eef-6927-4bad-a181-ef353a2553d4 None admin] RBAC: Adding query filter params (role.id=e0ab41d8ae22474688b5361bcc75e0f0, scope.project.id=86d84d636c124b04a6a1a0a18417a17f, user.id=06473bac23da480bb58714235678a732) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:01:34.874080 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d21eef-6927-4bad-a181-ef353a2553d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:01:34.875607 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d21eef-6927-4bad-a181-ef353a2553d4 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:01:34.882607 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 558/1146] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Sep 6 20:01:34 2022] GET /identity/v3/role_assignments?user.id=06473bac23da480bb58714235678a732&scope.project.id=86d84d636c124b04a6a1a0a18417a17f&role.id=e0ab41d8ae22474688b5361bcc75e0f0 => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 20:01:38.158874 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f25d93-a9c0-4995-a388-079e85228c41 None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:38.159526 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f25d93-a9c0-4995-a388-079e85228c41 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:38.159526 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f25d93-a9c0-4995-a388-079e85228c41 None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:38.160798 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 589/1147] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:01:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 20:01:38.167398 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df865432-632b-4a46-be8d-72558919e9e3 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:38.168003 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df865432-632b-4a46-be8d-72558919e9e3 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:38.168459 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df865432-632b-4a46-be8d-72558919e9e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:38.189932 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-df865432-632b-4a46-be8d-72558919e9e3 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:01:38.191171 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-df865432-632b-4a46-be8d-72558919e9e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:01:38.294117 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 559/1148] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:01:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 20:01:38.577260 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f25d93-a9c0-4995-a388-079e85228c41 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:01:38.651910 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:01:38.734292 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f25d93-a9c0-4995-a388-079e85228c41 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:01:38.801497 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351caea5-f76f-47be-87d7-c7d86ca332af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:01:38.802650 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351caea5-f76f-47be-87d7-c7d86ca332af None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:38.803165 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351caea5-f76f-47be-87d7-c7d86ca332af None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:38.803671 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351caea5-f76f-47be-87d7-c7d86ca332af None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:38.824719 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351caea5-f76f-47be-87d7-c7d86ca332af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 20:01:38.834893 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-351caea5-f76f-47be-87d7-c7d86ca332af None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 20:01:38.834893 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:01:38.834893 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:01:38.834893 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:01:38.834893 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:01:38.834893 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:01:38.834893 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:01:38.834893 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:01:38.834893 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:01:38.834893 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:01:38.834893 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:01:38.834893 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:01:38.834893 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 20:01:38.834893 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 20:01:38.834893 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 20:01:38.834893 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 20:01:38.836159 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 20:01:38.836159 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 20:01:38.836159 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 20:01:38.836159 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 20:01:38.836159 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:01:38.836159 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:01:38.836159 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:01:38.836159 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:01:38.836159 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:01:38.836159 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:01:38.836159 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:01:38.836159 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 20:01:38.836159 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:01:38.836159 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:01:38.836159 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:01:38.836159 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:01:38.837351 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:01:38.837351 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:01:38.837351 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:01:38.837351 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:01:38.837351 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 20:01:38.837351 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 20:01:38.837351 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 20:01:38.837351 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 20:01:38.837351 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 20:01:38.837351 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 20:01:38.837351 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 20:01:38.837351 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 20:01:38.837351 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 590/1149] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Sep 6 20:01:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:01:38.844337 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df865432-632b-4a46-be8d-72558919e9e3 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:01:38.922255 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:01:38.970798 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df865432-632b-4a46-be8d-72558919e9e3 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:01:39.013237 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12631dcd-bcbe-46f2-bbe4-1bbebad029e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:01:39.013929 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12631dcd-bcbe-46f2-bbe4-1bbebad029e2 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:39.014237 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12631dcd-bcbe-46f2-bbe4-1bbebad029e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:39.014769 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12631dcd-bcbe-46f2-bbe4-1bbebad029e2 None admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:39.015474 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12631dcd-bcbe-46f2-bbe4-1bbebad029e2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:01:39.015781 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12631dcd-bcbe-46f2-bbe4-1bbebad029e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:01:39.017201 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12631dcd-bcbe-46f2-bbe4-1bbebad029e2 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:01:39.027385 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 560/1150] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 6 20:01:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:01:39.040541 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351caea5-f76f-47be-87d7-c7d86ca332af None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:01:39.163312 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351caea5-f76f-47be-87d7-c7d86ca332af None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:01:39.184503 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:01:39.207080 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414e3677-65aa-4c8d-847f-7d551487830d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:01:39.208201 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414e3677-65aa-4c8d-847f-7d551487830d None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:39.208677 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414e3677-65aa-4c8d-847f-7d551487830d None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:39.208964 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414e3677-65aa-4c8d-847f-7d551487830d None admin] PATH_INFO: `/v3/users/06473bac23da480bb58714235678a732` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:39.210322 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414e3677-65aa-4c8d-847f-7d551487830d None admin] RBAC: Authorizing `identity:get_user(user_id=06473bac23da480bb58714235678a732)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:01:39.211830 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414e3677-65aa-4c8d-847f-7d551487830d None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:01:39.213572 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 591/1151] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 6 20:01:39 2022] GET /identity/v3/users/06473bac23da480bb58714235678a732 => generated 286 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:01:39.220780 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12631dcd-bcbe-46f2-bbe4-1bbebad029e2 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:01:39.350760 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12631dcd-bcbe-46f2-bbe4-1bbebad029e2 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:01:39.379147 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:01:39.397093 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834ae716-da70-42dc-b4a5-a9b2377c2d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:01:39.397093 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834ae716-da70-42dc-b4a5-a9b2377c2d22 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:39.397093 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834ae716-da70-42dc-b4a5-a9b2377c2d22 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:39.397093 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834ae716-da70-42dc-b4a5-a9b2377c2d22 None admin] PATH_INFO: `/v3/projects/86d84d636c124b04a6a1a0a18417a17f` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:39.397869 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834ae716-da70-42dc-b4a5-a9b2377c2d22 None admin] RBAC: Authorizing `identity:get_project(project_id=86d84d636c124b04a6a1a0a18417a17f)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:01:39.397869 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834ae716-da70-42dc-b4a5-a9b2377c2d22 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:01:39.398249 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 561/1152] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 6 20:01:39 2022] GET /identity/v3/projects/86d84d636c124b04a6a1a0a18417a17f => generated 311 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:01:39.404708 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414e3677-65aa-4c8d-847f-7d551487830d None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:01:39.535508 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414e3677-65aa-4c8d-847f-7d551487830d None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:01:39.574328 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708bc6be-e8d2-4e2c-99c1-ae00f3ad2132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:01:39.575045 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708bc6be-e8d2-4e2c-99c1-ae00f3ad2132 None admin] REQUEST_METHOD: `PUT` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:39.575917 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708bc6be-e8d2-4e2c-99c1-ae00f3ad2132 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:39.575917 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708bc6be-e8d2-4e2c-99c1-ae00f3ad2132 None admin] PATH_INFO: `/v3/projects/86d84d636c124b04a6a1a0a18417a17f/users/06473bac23da480bb58714235678a732/roles/e0ab41d8ae22474688b5361bcc75e0f0` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:39.577652 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708bc6be-e8d2-4e2c-99c1-ae00f3ad2132 None admin] RBAC: Authorizing `identity:create_grant(project_id=86d84d636c124b04a6a1a0a18417a17f, user_id=06473bac23da480bb58714235678a732, role_id=e0ab41d8ae22474688b5361bcc75e0f0)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:01:39.579292 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708bc6be-e8d2-4e2c-99c1-ae00f3ad2132 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:01:39.595140 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 592/1153] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Sep 6 20:01:39 2022] PUT /identity/v3/projects/86d84d636c124b04a6a1a0a18417a17f/users/06473bac23da480bb58714235678a732/roles/e0ab41d8ae22474688b5361bcc75e0f0 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:01:41.921370 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30870661-c1cf-47f5-a1ab-c711294627dc None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:41.921982 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30870661-c1cf-47f5-a1ab-c711294627dc None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:41.921982 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30870661-c1cf-47f5-a1ab-c711294627dc None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:41.923363 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 562/1154] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:01:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:01:41.933228 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d5cd9b-d106-4bea-b635-bd5363ae246d None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:41.933840 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d5cd9b-d106-4bea-b635-bd5363ae246d None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:41.934777 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d5cd9b-d106-4bea-b635-bd5363ae246d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:41.968116 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-f2d5cd9b-d106-4bea-b635-bd5363ae246d None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:01:41.970249 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-f2d5cd9b-d106-4bea-b635-bd5363ae246d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:01:42.052596 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:01:42.091755 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 593/1155] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:01:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 20:01:42.373313 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30870661-c1cf-47f5-a1ab-c711294627dc None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:01:42.503193 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30870661-c1cf-47f5-a1ab-c711294627dc None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:01:42.544387 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7fa7c2-2a0a-4089-b7db-03357ac267bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:01:42.545039 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7fa7c2-2a0a-4089-b7db-03357ac267bc None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:42.545324 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7fa7c2-2a0a-4089-b7db-03357ac267bc None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:42.545603 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7fa7c2-2a0a-4089-b7db-03357ac267bc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:42.559515 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7fa7c2-2a0a-4089-b7db-03357ac267bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 20:01:42.564627 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-4f7fa7c2-2a0a-4089-b7db-03357ac267bc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 20:01:42.564627 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:01:42.564627 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:01:42.564627 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:01:42.564627 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:01:42.564627 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:01:42.564627 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:01:42.564627 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:01:42.564627 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:01:42.564627 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:01:42.564627 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:01:42.564627 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:01:42.564627 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 20:01:42.564627 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 20:01:42.564627 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 20:01:42.564627 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 20:01:42.566622 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 20:01:42.566622 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 20:01:42.566622 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 20:01:42.566622 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 20:01:42.566622 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:01:42.566622 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:01:42.566622 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:01:42.566622 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:01:42.566622 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:01:42.566622 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:01:42.566622 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:01:42.566622 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 20:01:42.566622 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:01:42.566622 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:01:42.566622 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:01:42.566622 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:01:42.568593 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:01:42.568593 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:01:42.568593 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:01:42.568593 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:01:42.568593 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 20:01:42.568593 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 20:01:42.568593 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 20:01:42.568593 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 20:01:42.568593 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 20:01:42.568593 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 20:01:42.568593 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 20:01:42.568593 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:01:42.568593 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 563/1156] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Sep 6 20:01:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:01:42.574210 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d5cd9b-d106-4bea-b635-bd5363ae246d None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:01:42.695417 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d5cd9b-d106-4bea-b635-bd5363ae246d None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:01:42.737639 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5416c4-817c-41f3-a1ae-ccb320be2add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:01:42.738269 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5416c4-817c-41f3-a1ae-ccb320be2add None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:42.738557 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5416c4-817c-41f3-a1ae-ccb320be2add None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:42.738838 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5416c4-817c-41f3-a1ae-ccb320be2add None admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:42.739535 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5416c4-817c-41f3-a1ae-ccb320be2add None admin] RBAC: Adding query filter params (name=admin) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:01:42.739872 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5416c4-817c-41f3-a1ae-ccb320be2add None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:01:42.741263 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5416c4-817c-41f3-a1ae-ccb320be2add None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:01:42.751853 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 594/1157] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 6 20:01:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:01:42.758647 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7fa7c2-2a0a-4089-b7db-03357ac267bc None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:01:42.821261 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:01:42.881494 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7fa7c2-2a0a-4089-b7db-03357ac267bc None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:01:42.925631 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f06766-0435-483b-8f16-267cf62e461f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:01:42.926318 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f06766-0435-483b-8f16-267cf62e461f None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:42.926630 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f06766-0435-483b-8f16-267cf62e461f None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:42.926921 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f06766-0435-483b-8f16-267cf62e461f None admin] PATH_INFO: `/v3/users/06473bac23da480bb58714235678a732` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:42.928228 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f06766-0435-483b-8f16-267cf62e461f None admin] RBAC: Authorizing `identity:get_user(user_id=06473bac23da480bb58714235678a732)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:01:42.929815 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f06766-0435-483b-8f16-267cf62e461f None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:01:42.931581 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 564/1158] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 6 20:01:42 2022] GET /identity/v3/users/06473bac23da480bb58714235678a732 => generated 286 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 20:01:42.938351 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5416c4-817c-41f3-a1ae-ccb320be2add None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:01:42.995477 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:01:43.098607 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5416c4-817c-41f3-a1ae-ccb320be2add None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:01:43.142574 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bde0410-6ff9-4c38-b896-ce5468ade180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:01:43.143476 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bde0410-6ff9-4c38-b896-ce5468ade180 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:43.143809 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bde0410-6ff9-4c38-b896-ce5468ade180 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:43.144132 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bde0410-6ff9-4c38-b896-ce5468ade180 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:43.229398 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bde0410-6ff9-4c38-b896-ce5468ade180 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:01:43.230954 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bde0410-6ff9-4c38-b896-ce5468ade180 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:01:43.276536 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:01:43.351892 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 595/1159] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 6 20:01:42 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 06 20:01:43.358516 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f06766-0435-483b-8f16-267cf62e461f None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:01:43.530976 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f06766-0435-483b-8f16-267cf62e461f None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:01:43.569866 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:01:43.595573 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a319905b-cbcd-4405-9a9b-4f01283d27e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:01:43.596564 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a319905b-cbcd-4405-9a9b-4f01283d27e8 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:43.597008 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a319905b-cbcd-4405-9a9b-4f01283d27e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:43.597471 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a319905b-cbcd-4405-9a9b-4f01283d27e8 None admin] PATH_INFO: `/v3/projects/86d84d636c124b04a6a1a0a18417a17f` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:43.599645 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a319905b-cbcd-4405-9a9b-4f01283d27e8 None admin] RBAC: Authorizing `identity:get_project(project_id=86d84d636c124b04a6a1a0a18417a17f)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:01:43.602393 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a319905b-cbcd-4405-9a9b-4f01283d27e8 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:01:43.605079 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 565/1160] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 6 20:01:43 2022] GET /identity/v3/projects/86d84d636c124b04a6a1a0a18417a17f => generated 311 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:01:43.612913 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bde0410-6ff9-4c38-b896-ce5468ade180 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:01:43.742192 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bde0410-6ff9-4c38-b896-ce5468ade180 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:01:43.782953 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e1a932-439d-4c94-875c-7b98083fab00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:01:43.783612 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e1a932-439d-4c94-875c-7b98083fab00 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:43.783877 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e1a932-439d-4c94-875c-7b98083fab00 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:43.784152 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e1a932-439d-4c94-875c-7b98083fab00 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:43.785042 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e1a932-439d-4c94-875c-7b98083fab00 None admin] RBAC: Adding query filter params (role.id=e0ab41d8ae22474688b5361bcc75e0f0, scope.project.id=86d84d636c124b04a6a1a0a18417a17f, user.id=06473bac23da480bb58714235678a732) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:01:43.785406 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e1a932-439d-4c94-875c-7b98083fab00 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:01:43.786967 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e1a932-439d-4c94-875c-7b98083fab00 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:01:43.793987 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 596/1161] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Sep 6 20:01:43 2022] GET /identity/v3/role_assignments?user.id=06473bac23da480bb58714235678a732&scope.project.id=86d84d636c124b04a6a1a0a18417a17f&role.id=e0ab41d8ae22474688b5361bcc75e0f0 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:01:46.252334 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f28498-26f8-4a78-8beb-753d5e99508d None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:46.252959 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f28498-26f8-4a78-8beb-753d5e99508d None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:46.253307 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f28498-26f8-4a78-8beb-753d5e99508d None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:46.255899 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 566/1162] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:01:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:01:46.262142 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcd5c75-6062-426a-9ca4-11805fc999f9 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:46.262937 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcd5c75-6062-426a-9ca4-11805fc999f9 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:46.263091 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcd5c75-6062-426a-9ca4-11805fc999f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:46.270110 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:01:46.291608 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-9bcd5c75-6062-426a-9ca4-11805fc999f9 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:01:46.293898 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-9bcd5c75-6062-426a-9ca4-11805fc999f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:01:46.431400 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 597/1163] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:01:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 20:01:46.712399 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f28498-26f8-4a78-8beb-753d5e99508d None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:01:46.836959 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f28498-26f8-4a78-8beb-753d5e99508d None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:01:46.885457 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3063b7-16a9-44d7-be50-9621a51de696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:01:46.886183 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3063b7-16a9-44d7-be50-9621a51de696 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:46.886510 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3063b7-16a9-44d7-be50-9621a51de696 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:46.886827 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3063b7-16a9-44d7-be50-9621a51de696 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:46.957953 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:01:46.974542 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3063b7-16a9-44d7-be50-9621a51de696 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:01:46.976284 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3063b7-16a9-44d7-be50-9621a51de696 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:01:47.104606 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 567/1164] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 6 20:01:46 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 06 20:01:47.111540 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcd5c75-6062-426a-9ca4-11805fc999f9 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:01:47.234021 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcd5c75-6062-426a-9ca4-11805fc999f9 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:01:47.270841 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:01:47.279591 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d19df48-48b0-4a2b-85e5-061cb1bd74e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:01:47.280268 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d19df48-48b0-4a2b-85e5-061cb1bd74e5 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:47.280662 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d19df48-48b0-4a2b-85e5-061cb1bd74e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:47.280942 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d19df48-48b0-4a2b-85e5-061cb1bd74e5 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:47.290230 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d19df48-48b0-4a2b-85e5-061cb1bd74e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:01:47.293199 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d19df48-48b0-4a2b-85e5-061cb1bd74e5 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:01:47.316775 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-4d19df48-48b0-4a2b-85e5-061cb1bd74e5 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 06 20:01:47.316775 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:01:47.316775 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 06 20:01:47.316775 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 06 20:01:47.316775 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 20:01:47.316775 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 20:01:47.316775 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 20:01:47.316775 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 20:01:47.316775 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 06 20:01:47.316775 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 20:01:47.316775 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 20:01:47.316775 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 20:01:47.316775 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:01:47.316775 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:01:47.316775 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:01:47.316775 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:01:47.316775 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:01:47.318662 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:01:47.318662 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:01:47.318662 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:01:47.318662 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:01:47.318662 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:01:47.318662 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:01:47.318662 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 20:01:47.318662 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 20:01:47.318662 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 20:01:47.318662 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 20:01:47.318662 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:01:47.318662 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:01:47.318662 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:01:47.318662 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:01:47.318662 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:01:47.318662 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:01:47.319718 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:01:47.319718 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 20:01:47.319718 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:01:47.319718 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:01:47.319718 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:01:47.319718 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:01:47.319718 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:01:47.319718 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:01:47.319718 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:01:47.319718 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:01:47.319718 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 06 20:01:47.319718 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 20:01:47.319718 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 06 20:01:47.319718 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 20:01:47.319718 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 598/1165] 10.222.0.20 () {56 vars in 1117 bytes} [Tue Sep 6 20:01:47 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:01:47.328454 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3063b7-16a9-44d7-be50-9621a51de696 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:01:47.467591 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:01:47.489899 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3063b7-16a9-44d7-be50-9621a51de696 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:01:47.553991 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f522f3-7e63-4a39-bd0a-f97c7e08a4dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:01:47.555041 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f522f3-7e63-4a39-bd0a-f97c7e08a4dd None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:47.555529 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f522f3-7e63-4a39-bd0a-f97c7e08a4dd None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:47.556004 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f522f3-7e63-4a39-bd0a-f97c7e08a4dd None admin] PATH_INFO: `/v3/domains` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:47.557245 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f522f3-7e63-4a39-bd0a-f97c7e08a4dd None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:01:47.557898 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f522f3-7e63-4a39-bd0a-f97c7e08a4dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:01:47.561337 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f522f3-7e63-4a39-bd0a-f97c7e08a4dd None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:01:47.595013 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 568/1166] 10.222.0.20 () {56 vars in 1115 bytes} [Tue Sep 6 20:01:47 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:01:50.638436 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328b283c-ccce-4a11-a621-dd0b9ef571a5 None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:50.639040 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328b283c-ccce-4a11-a621-dd0b9ef571a5 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:50.639391 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328b283c-ccce-4a11-a621-dd0b9ef571a5 None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:50.642202 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 599/1167] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:01:50 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:01:50.650670 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d91b0e-f1cb-4827-a202-080269256089 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:50.651244 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d91b0e-f1cb-4827-a202-080269256089 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:50.651244 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d91b0e-f1cb-4827-a202-080269256089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:50.674950 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-25d91b0e-f1cb-4827-a202-080269256089 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:01:50.676623 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-25d91b0e-f1cb-4827-a202-080269256089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:01:50.763368 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 569/1168] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:01:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 20:01:51.065210 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328b283c-ccce-4a11-a621-dd0b9ef571a5 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:01:51.252823 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328b283c-ccce-4a11-a621-dd0b9ef571a5 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:01:51.318456 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca81d6ed-5ccf-4853-b893-cff997107e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:01:51.319410 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca81d6ed-5ccf-4853-b893-cff997107e57 None admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:51.319838 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca81d6ed-5ccf-4853-b893-cff997107e57 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:51.320236 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca81d6ed-5ccf-4853-b893-cff997107e57 None admin] PATH_INFO: `/v3/domains` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:51.321273 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca81d6ed-5ccf-4853-b893-cff997107e57 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:01:51.323540 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca81d6ed-5ccf-4853-b893-cff997107e57 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:01:51.346102 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 600/1169] 10.222.0.20 () {60 vars in 1138 bytes} [Tue Sep 6 20:01:51 2022] POST /identity/v3/domains => generated 267 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:01:53.767008 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cfbab4-4ced-401b-9f3e-c7e21318d821 None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:53.767561 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cfbab4-4ced-401b-9f3e-c7e21318d821 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:53.767728 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cfbab4-4ced-401b-9f3e-c7e21318d821 None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:53.769109 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 570/1170] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:01:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 20:01:53.778042 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a81ba8a-b47f-4886-96a4-8b64d9a129c8 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:53.779336 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a81ba8a-b47f-4886-96a4-8b64d9a129c8 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:53.779609 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a81ba8a-b47f-4886-96a4-8b64d9a129c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:53.799551 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-9a81ba8a-b47f-4886-96a4-8b64d9a129c8 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:01:53.800811 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-9a81ba8a-b47f-4886-96a4-8b64d9a129c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:01:53.859934 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:01:53.887333 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 601/1171] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:01:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 20:01:54.163382 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cfbab4-4ced-401b-9f3e-c7e21318d821 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:01:54.274997 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:01:54.301221 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cfbab4-4ced-401b-9f3e-c7e21318d821 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:01:54.341949 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f73412-15ea-4107-9ff0-8fd23ee02f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:01:54.342606 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f73412-15ea-4107-9ff0-8fd23ee02f4d None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:54.342979 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f73412-15ea-4107-9ff0-8fd23ee02f4d None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:54.343261 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f73412-15ea-4107-9ff0-8fd23ee02f4d None admin] PATH_INFO: `/v3/domains/cb3b15e22a3145c6b544bbc547ae3dba` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:54.357176 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f73412-15ea-4107-9ff0-8fd23ee02f4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=cb3b15e22a3145c6b544bbc547ae3dba)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:01:54.359372 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f73412-15ea-4107-9ff0-8fd23ee02f4d None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:01:54.361305 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 571/1172] 10.222.0.20 () {56 vars in 1183 bytes} [Tue Sep 6 20:01:54 2022] GET /identity/v3/domains/cb3b15e22a3145c6b544bbc547ae3dba => generated 267 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 20:01:54.368579 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a81ba8a-b47f-4886-96a4-8b64d9a129c8 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:01:54.489466 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a81ba8a-b47f-4886-96a4-8b64d9a129c8 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:01:54.529489 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cd762c-04b9-4592-aa33-fe6b15cf05bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:01:54.530147 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cd762c-04b9-4592-aa33-fe6b15cf05bb None admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:54.530425 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cd762c-04b9-4592-aa33-fe6b15cf05bb None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:54.530690 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cd762c-04b9-4592-aa33-fe6b15cf05bb None admin] PATH_INFO: `/v3/projects` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:54.531358 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cd762c-04b9-4592-aa33-fe6b15cf05bb None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:01:54.532837 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cd762c-04b9-4592-aa33-fe6b15cf05bb None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:01:54.565303 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 602/1173] 10.222.0.20 () {60 vars in 1141 bytes} [Tue Sep 6 20:01:54 2022] POST /identity/v3/projects => generated 361 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:01:57.008997 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242a2c32-0bfa-4fcf-a29f-77a8e0a37f15 None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:57.009941 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242a2c32-0bfa-4fcf-a29f-77a8e0a37f15 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:57.009941 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242a2c32-0bfa-4fcf-a29f-77a8e0a37f15 None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:57.011518 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 572/1174] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:01:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 20:01:57.018899 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617acc2a-0ce6-43f0-9b79-d67efab7f3f9 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:57.019600 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617acc2a-0ce6-43f0-9b79-d67efab7f3f9 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:57.019600 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617acc2a-0ce6-43f0-9b79-d67efab7f3f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:57.040650 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-617acc2a-0ce6-43f0-9b79-d67efab7f3f9 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:01:57.042004 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-617acc2a-0ce6-43f0-9b79-d67efab7f3f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:01:57.076379 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:01:57.125763 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 603/1175] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:01:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 20:01:57.403171 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242a2c32-0bfa-4fcf-a29f-77a8e0a37f15 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:01:57.530940 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242a2c32-0bfa-4fcf-a29f-77a8e0a37f15 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:01:57.571650 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c850f44-c591-4bbb-a92c-72f374fcf04e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:01:57.572412 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c850f44-c591-4bbb-a92c-72f374fcf04e None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:57.572749 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c850f44-c591-4bbb-a92c-72f374fcf04e None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:57.573061 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c850f44-c591-4bbb-a92c-72f374fcf04e None admin] PATH_INFO: `/v3/domains/cb3b15e22a3145c6b544bbc547ae3dba` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:57.574366 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c850f44-c591-4bbb-a92c-72f374fcf04e None admin] RBAC: Authorizing `identity:get_domain(domain_id=cb3b15e22a3145c6b544bbc547ae3dba)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:01:57.575992 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c850f44-c591-4bbb-a92c-72f374fcf04e None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:01:57.577957 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 573/1176] 10.222.0.20 () {56 vars in 1183 bytes} [Tue Sep 6 20:01:57 2022] GET /identity/v3/domains/cb3b15e22a3145c6b544bbc547ae3dba => generated 267 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 20:01:57.587002 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617acc2a-0ce6-43f0-9b79-d67efab7f3f9 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:01:57.709262 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617acc2a-0ce6-43f0-9b79-d67efab7f3f9 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:01:57.751671 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee9dd95-5b12-4034-a21b-e5e5401685e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:01:57.752305 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee9dd95-5b12-4034-a21b-e5e5401685e2 None admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:01:57.752587 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee9dd95-5b12-4034-a21b-e5e5401685e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:01:57.752864 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee9dd95-5b12-4034-a21b-e5e5401685e2 None admin] PATH_INFO: `/v3/users` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:01:57.753563 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee9dd95-5b12-4034-a21b-e5e5401685e2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:01:57.755111 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee9dd95-5b12-4034-a21b-e5e5401685e2 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:01:57.782748 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 604/1177] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Sep 6 20:01:57 2022] POST /identity/v3/users => generated 311 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:02:00.521980 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3dca90-7f6f-4593-94f8-6797e8a4fca6 None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:02:00.522836 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3dca90-7f6f-4593-94f8-6797e8a4fca6 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:02:00.522836 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3dca90-7f6f-4593-94f8-6797e8a4fca6 None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:02:00.524423 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 574/1178] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:02:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:02:00.531176 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81350e61-10af-47a0-9658-f4c6c1204123 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:02:00.531797 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81350e61-10af-47a0-9658-f4c6c1204123 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:02:00.531797 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81350e61-10af-47a0-9658-f4c6c1204123 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:02:00.560873 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-81350e61-10af-47a0-9658-f4c6c1204123 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:02:00.563091 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-81350e61-10af-47a0-9658-f4c6c1204123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:02:00.592419 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:02:00.694288 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 605/1179] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:02:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 20:02:00.981575 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3dca90-7f6f-4593-94f8-6797e8a4fca6 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:02:00.998006 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:02:01.116991 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3dca90-7f6f-4593-94f8-6797e8a4fca6 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:02:01.177026 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0460679-02a8-4786-b3a7-da7f84bb7229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:02:01.177852 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0460679-02a8-4786-b3a7-da7f84bb7229 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:02:01.178150 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0460679-02a8-4786-b3a7-da7f84bb7229 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:02:01.178543 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0460679-02a8-4786-b3a7-da7f84bb7229 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:02:01.192925 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0460679-02a8-4786-b3a7-da7f84bb7229 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 20:02:01.198483 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-e0460679-02a8-4786-b3a7-da7f84bb7229 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 20:02:01.198483 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:02:01.198483 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:02:01.198483 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:02:01.198483 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:02:01.198483 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:02:01.198483 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:02:01.198483 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:02:01.198483 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:02:01.198483 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:02:01.198483 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:02:01.198483 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:02:01.198483 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 20:02:01.198483 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 20:02:01.198483 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 20:02:01.198483 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 20:02:01.200476 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 20:02:01.200476 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 20:02:01.200476 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 20:02:01.200476 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 20:02:01.200476 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:02:01.200476 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:02:01.200476 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:02:01.200476 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:02:01.200476 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:02:01.200476 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:02:01.200476 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:02:01.200476 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 20:02:01.200476 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:02:01.200476 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:02:01.200476 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:02:01.200476 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:02:01.201778 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:02:01.201778 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:02:01.201778 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:02:01.201778 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:02:01.201778 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 20:02:01.201778 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 20:02:01.201778 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 20:02:01.201778 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 20:02:01.201778 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 20:02:01.201778 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 20:02:01.201778 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 20:02:01.201778 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:02:01.201778 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 575/1180] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Sep 6 20:02:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 20:02:01.207856 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81350e61-10af-47a0-9658-f4c6c1204123 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:02:01.325547 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81350e61-10af-47a0-9658-f4c6c1204123 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:02:01.375363 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b1b520-8b09-465a-93ac-3427bea5fb14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:02:01.376351 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b1b520-8b09-465a-93ac-3427bea5fb14 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:02:01.376816 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b1b520-8b09-465a-93ac-3427bea5fb14 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:02:01.377102 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b1b520-8b09-465a-93ac-3427bea5fb14 None admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:02:01.377967 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b1b520-8b09-465a-93ac-3427bea5fb14 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:02:01.378309 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b1b520-8b09-465a-93ac-3427bea5fb14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:02:01.380302 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b1b520-8b09-465a-93ac-3427bea5fb14 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:02:01.395776 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 606/1181] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 6 20:02:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 20:02:01.403027 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0460679-02a8-4786-b3a7-da7f84bb7229 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:02:01.491250 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:02:01.526312 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0460679-02a8-4786-b3a7-da7f84bb7229 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:02:01.567853 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6322036-2585-4993-9b36-532e550dca5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:02:01.568656 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6322036-2585-4993-9b36-532e550dca5a None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:02:01.569041 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6322036-2585-4993-9b36-532e550dca5a None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:02:01.569407 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6322036-2585-4993-9b36-532e550dca5a None admin] PATH_INFO: `/v3/users/962de5f937b0457a8e79f0f91cb3e40c` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:02:01.588756 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6322036-2585-4993-9b36-532e550dca5a None admin] RBAC: Authorizing `identity:get_user(user_id=962de5f937b0457a8e79f0f91cb3e40c)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:02:01.590345 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6322036-2585-4993-9b36-532e550dca5a None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:02:01.595429 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 576/1182] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 6 20:02:01 2022] GET /identity/v3/users/962de5f937b0457a8e79f0f91cb3e40c => generated 311 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:02:01.601999 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b1b520-8b09-465a-93ac-3427bea5fb14 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:02:01.620680 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:02:01.737598 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b1b520-8b09-465a-93ac-3427bea5fb14 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:02:01.778770 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cee2cd-eb09-4881-bc35-aa92483287cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:02:01.779919 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cee2cd-eb09-4881-bc35-aa92483287cc None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:02:01.780192 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cee2cd-eb09-4881-bc35-aa92483287cc None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:02:01.780460 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cee2cd-eb09-4881-bc35-aa92483287cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:02:01.861235 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cee2cd-eb09-4881-bc35-aa92483287cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:02:01.862807 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cee2cd-eb09-4881-bc35-aa92483287cc None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:02:01.880301 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:02:01.992193 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 607/1183] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 6 20:02:01 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 20:02:01.999158 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6322036-2585-4993-9b36-532e550dca5a None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:02:02.126492 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6322036-2585-4993-9b36-532e550dca5a None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:02:02.160529 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:02:02.170851 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106c635a-340c-4fb1-a30b-81aa4db3b297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:02:02.171504 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106c635a-340c-4fb1-a30b-81aa4db3b297 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:02:02.171780 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106c635a-340c-4fb1-a30b-81aa4db3b297 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:02:02.172050 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106c635a-340c-4fb1-a30b-81aa4db3b297 None admin] PATH_INFO: `/v3/projects/5b4a260355444065be7ad89fe4339976` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:02:02.173337 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106c635a-340c-4fb1-a30b-81aa4db3b297 None admin] RBAC: Authorizing `identity:get_project(project_id=5b4a260355444065be7ad89fe4339976)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:02:02.174846 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106c635a-340c-4fb1-a30b-81aa4db3b297 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:02:02.176594 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 577/1184] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 6 20:02:01 2022] GET /identity/v3/projects/5b4a260355444065be7ad89fe4339976 => generated 361 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:02:02.183026 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cee2cd-eb09-4881-bc35-aa92483287cc None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:02:02.303158 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cee2cd-eb09-4881-bc35-aa92483287cc None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:02:02.318157 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:02:02.355204 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db9d577-6e38-4a57-8321-6485a2607905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:02:02.356153 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db9d577-6e38-4a57-8321-6485a2607905 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:02:02.356601 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db9d577-6e38-4a57-8321-6485a2607905 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:02:02.357027 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db9d577-6e38-4a57-8321-6485a2607905 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:02:02.358352 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db9d577-6e38-4a57-8321-6485a2607905 None admin] RBAC: Adding query filter params (role.id=e0ab41d8ae22474688b5361bcc75e0f0, scope.project.id=5b4a260355444065be7ad89fe4339976, user.id=962de5f937b0457a8e79f0f91cb3e40c) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:02:02.358861 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db9d577-6e38-4a57-8321-6485a2607905 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:02:02.361303 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db9d577-6e38-4a57-8321-6485a2607905 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:02:02.371029 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 608/1185] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Sep 6 20:02:02 2022] GET /identity/v3/role_assignments?user.id=962de5f937b0457a8e79f0f91cb3e40c&scope.project.id=5b4a260355444065be7ad89fe4339976&role.id=e0ab41d8ae22474688b5361bcc75e0f0 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 20:02:04.712306 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef10cefa-38cf-451b-9f52-1a58d5c52771 None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:02:04.712926 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef10cefa-38cf-451b-9f52-1a58d5c52771 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:02:04.712926 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef10cefa-38cf-451b-9f52-1a58d5c52771 None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:02:04.714457 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 578/1186] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:02:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:02:04.721906 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ff78bb-f957-462e-81ed-b8e1f76e0c6d None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:02:04.722722 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ff78bb-f957-462e-81ed-b8e1f76e0c6d None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:02:04.722722 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ff78bb-f957-462e-81ed-b8e1f76e0c6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:02:04.743331 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-19ff78bb-f957-462e-81ed-b8e1f76e0c6d None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:02:04.744683 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-19ff78bb-f957-462e-81ed-b8e1f76e0c6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:02:04.829044 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 609/1187] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:02:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 20:02:05.101910 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef10cefa-38cf-451b-9f52-1a58d5c52771 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:02:05.227490 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef10cefa-38cf-451b-9f52-1a58d5c52771 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:02:05.272673 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82203a4-806c-434b-b632-59b910dea931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:02:05.273323 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82203a4-806c-434b-b632-59b910dea931 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:02:05.273800 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82203a4-806c-434b-b632-59b910dea931 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:02:05.273947 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82203a4-806c-434b-b632-59b910dea931 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:02:05.287663 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82203a4-806c-434b-b632-59b910dea931 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 20:02:05.293056 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-a82203a4-806c-434b-b632-59b910dea931 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 20:02:05.293056 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:02:05.293056 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:02:05.293056 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:02:05.293056 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:02:05.293056 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:02:05.293056 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:02:05.293056 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:02:05.293056 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:02:05.293056 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:02:05.293056 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:02:05.293056 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:02:05.293056 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 20:02:05.293056 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 20:02:05.293056 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 20:02:05.293056 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 20:02:05.294914 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 20:02:05.294914 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 20:02:05.294914 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 20:02:05.294914 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 20:02:05.294914 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:02:05.294914 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:02:05.294914 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:02:05.294914 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:02:05.294914 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:02:05.294914 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:02:05.294914 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:02:05.294914 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 20:02:05.294914 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:02:05.294914 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:02:05.294914 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:02:05.294914 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:02:05.297065 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:02:05.297065 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:02:05.297065 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:02:05.297065 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:02:05.297065 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 20:02:05.297065 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 20:02:05.297065 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 20:02:05.297065 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 20:02:05.297065 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 20:02:05.297065 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 20:02:05.297065 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 20:02:05.297065 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:02:05.297065 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 579/1188] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Sep 6 20:02:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 20:02:05.301871 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ff78bb-f957-462e-81ed-b8e1f76e0c6d None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:02:05.426068 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ff78bb-f957-462e-81ed-b8e1f76e0c6d None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:02:05.435220 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:02:05.477923 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9904fd2-b213-4a5b-ab26-39a6dc723e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:02:05.483139 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9904fd2-b213-4a5b-ab26-39a6dc723e14 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:02:05.483459 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9904fd2-b213-4a5b-ab26-39a6dc723e14 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:02:05.483740 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9904fd2-b213-4a5b-ab26-39a6dc723e14 None admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:02:05.484550 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9904fd2-b213-4a5b-ab26-39a6dc723e14 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:02:05.485023 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9904fd2-b213-4a5b-ab26-39a6dc723e14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:02:05.486812 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9904fd2-b213-4a5b-ab26-39a6dc723e14 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:02:05.497360 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 610/1189] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 6 20:02:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:02:05.508654 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82203a4-806c-434b-b632-59b910dea931 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:02:05.585343 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:02:05.635008 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82203a4-806c-434b-b632-59b910dea931 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:02:05.696554 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b356aef4-64cb-4988-89f4-052f347fbe07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:02:05.697537 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b356aef4-64cb-4988-89f4-052f347fbe07 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:02:05.698057 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b356aef4-64cb-4988-89f4-052f347fbe07 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:02:05.698531 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b356aef4-64cb-4988-89f4-052f347fbe07 None admin] PATH_INFO: `/v3/users/962de5f937b0457a8e79f0f91cb3e40c` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:02:05.700692 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b356aef4-64cb-4988-89f4-052f347fbe07 None admin] RBAC: Authorizing `identity:get_user(user_id=962de5f937b0457a8e79f0f91cb3e40c)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:02:05.703518 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b356aef4-64cb-4988-89f4-052f347fbe07 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:02:05.706717 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 580/1190] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 6 20:02:05 2022] GET /identity/v3/users/962de5f937b0457a8e79f0f91cb3e40c => generated 311 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:02:05.713178 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9904fd2-b213-4a5b-ab26-39a6dc723e14 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:02:05.840114 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9904fd2-b213-4a5b-ab26-39a6dc723e14 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:02:05.878815 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cee88a6-a7c0-4ded-8650-582ab63f99fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:02:05.879492 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cee88a6-a7c0-4ded-8650-582ab63f99fa None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:02:05.879781 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cee88a6-a7c0-4ded-8650-582ab63f99fa None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:02:05.880054 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cee88a6-a7c0-4ded-8650-582ab63f99fa None admin] PATH_INFO: `/v3/projects/5b4a260355444065be7ad89fe4339976` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:02:05.881386 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cee88a6-a7c0-4ded-8650-582ab63f99fa None admin] RBAC: Authorizing `identity:get_project(project_id=5b4a260355444065be7ad89fe4339976)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:02:05.883059 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cee88a6-a7c0-4ded-8650-582ab63f99fa None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:02:05.884860 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 611/1191] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 6 20:02:05 2022] GET /identity/v3/projects/5b4a260355444065be7ad89fe4339976 => generated 361 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 20:02:05.894524 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b356aef4-64cb-4988-89f4-052f347fbe07 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:02:06.052382 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b356aef4-64cb-4988-89f4-052f347fbe07 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:02:06.080097 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:02:06.093505 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3623832-005f-4f9e-a321-15ae8cb3e40b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:02:06.094289 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3623832-005f-4f9e-a321-15ae8cb3e40b None admin] REQUEST_METHOD: `PUT` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:02:06.094557 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3623832-005f-4f9e-a321-15ae8cb3e40b None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:02:06.094917 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3623832-005f-4f9e-a321-15ae8cb3e40b None admin] PATH_INFO: `/v3/projects/5b4a260355444065be7ad89fe4339976/users/962de5f937b0457a8e79f0f91cb3e40c/roles/e0ab41d8ae22474688b5361bcc75e0f0` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:02:06.097079 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3623832-005f-4f9e-a321-15ae8cb3e40b None admin] RBAC: Authorizing `identity:create_grant(project_id=5b4a260355444065be7ad89fe4339976, user_id=962de5f937b0457a8e79f0f91cb3e40c, role_id=e0ab41d8ae22474688b5361bcc75e0f0)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:02:06.098612 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3623832-005f-4f9e-a321-15ae8cb3e40b None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:02:06.114486 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 581/1192] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Sep 6 20:02:05 2022] PUT /identity/v3/projects/5b4a260355444065be7ad89fe4339976/users/962de5f937b0457a8e79f0f91cb3e40c/roles/e0ab41d8ae22474688b5361bcc75e0f0 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:02:08.462370 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26f3d0d-d319-4213-a2c1-691eacdea72c None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:02:08.463152 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26f3d0d-d319-4213-a2c1-691eacdea72c None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:02:08.463152 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26f3d0d-d319-4213-a2c1-691eacdea72c None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:02:08.464335 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 612/1193] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:02:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 20:02:08.471873 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f5168b-4779-40f0-8265-6a13d9780b3a None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:02:08.472488 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f5168b-4779-40f0-8265-6a13d9780b3a None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:02:08.472488 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f5168b-4779-40f0-8265-6a13d9780b3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:02:08.492039 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-15f5168b-4779-40f0-8265-6a13d9780b3a None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:02:08.493292 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-15f5168b-4779-40f0-8265-6a13d9780b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:02:08.576298 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 582/1194] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:02:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 20:02:08.855501 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26f3d0d-d319-4213-a2c1-691eacdea72c None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:02:08.900102 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:02:08.989918 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26f3d0d-d319-4213-a2c1-691eacdea72c None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:02:09.031944 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371c1e41-6622-40f3-8490-7f16a3b46005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:02:09.032613 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371c1e41-6622-40f3-8490-7f16a3b46005 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:02:09.032889 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371c1e41-6622-40f3-8490-7f16a3b46005 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:02:09.033281 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371c1e41-6622-40f3-8490-7f16a3b46005 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:02:09.046526 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371c1e41-6622-40f3-8490-7f16a3b46005 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 20:02:09.051672 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-371c1e41-6622-40f3-8490-7f16a3b46005 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 20:02:09.051672 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:02:09.051672 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:02:09.051672 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:02:09.051672 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:02:09.051672 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:02:09.051672 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:02:09.051672 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:02:09.051672 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:02:09.051672 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:02:09.051672 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:02:09.051672 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:02:09.051672 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 20:02:09.051672 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 20:02:09.051672 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 20:02:09.051672 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 20:02:09.053445 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 20:02:09.053445 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 20:02:09.053445 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 20:02:09.053445 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 20:02:09.053445 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:02:09.053445 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:02:09.053445 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:02:09.053445 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:02:09.053445 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:02:09.053445 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:02:09.053445 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:02:09.053445 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 20:02:09.053445 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:02:09.053445 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:02:09.053445 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:02:09.053445 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:02:09.054521 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:02:09.054521 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:02:09.054521 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:02:09.054521 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:02:09.054521 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 20:02:09.054521 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 20:02:09.054521 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 20:02:09.054521 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 20:02:09.054521 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 20:02:09.054521 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 20:02:09.054521 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 20:02:09.054521 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 20:02:09.054521 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 613/1195] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Sep 6 20:02:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:02:09.060468 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f5168b-4779-40f0-8265-6a13d9780b3a None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:02:09.243579 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f5168b-4779-40f0-8265-6a13d9780b3a None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:02:09.272576 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:02:09.302097 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d5ea0d-d946-4d79-af3d-699b4d992973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:02:09.302737 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d5ea0d-d946-4d79-af3d-699b4d992973 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:02:09.303017 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d5ea0d-d946-4d79-af3d-699b4d992973 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:02:09.303996 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d5ea0d-d946-4d79-af3d-699b4d992973 None admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:02:09.304164 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d5ea0d-d946-4d79-af3d-699b4d992973 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:02:09.304468 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d5ea0d-d946-4d79-af3d-699b4d992973 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:02:09.305970 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d5ea0d-d946-4d79-af3d-699b4d992973 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:02:09.316164 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 583/1196] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 6 20:02:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:02:09.326485 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371c1e41-6622-40f3-8490-7f16a3b46005 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:02:09.459185 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:02:09.466328 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371c1e41-6622-40f3-8490-7f16a3b46005 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:02:09.510845 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6448deb0-9935-419f-9bd4-e0b223e5ae10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:02:09.511442 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6448deb0-9935-419f-9bd4-e0b223e5ae10 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:02:09.511726 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6448deb0-9935-419f-9bd4-e0b223e5ae10 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:02:09.512050 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6448deb0-9935-419f-9bd4-e0b223e5ae10 None admin] PATH_INFO: `/v3/users/962de5f937b0457a8e79f0f91cb3e40c` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:02:09.513455 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6448deb0-9935-419f-9bd4-e0b223e5ae10 None admin] RBAC: Authorizing `identity:get_user(user_id=962de5f937b0457a8e79f0f91cb3e40c)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:02:09.515564 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6448deb0-9935-419f-9bd4-e0b223e5ae10 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:02:09.517264 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 614/1197] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 6 20:02:09 2022] GET /identity/v3/users/962de5f937b0457a8e79f0f91cb3e40c => generated 311 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:02:09.523838 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d5ea0d-d946-4d79-af3d-699b4d992973 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:02:09.695014 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d5ea0d-d946-4d79-af3d-699b4d992973 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:02:09.766755 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0ed9fa-19c0-4c51-9345-5b1630179e43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:02:09.767935 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0ed9fa-19c0-4c51-9345-5b1630179e43 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:02:09.768436 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0ed9fa-19c0-4c51-9345-5b1630179e43 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:02:09.768907 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0ed9fa-19c0-4c51-9345-5b1630179e43 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:02:09.830392 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:02:09.859882 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0ed9fa-19c0-4c51-9345-5b1630179e43 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:02:09.861496 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0ed9fa-19c0-4c51-9345-5b1630179e43 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:02:10.023371 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 584/1198] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 6 20:02:09 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 500 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 06 20:02:10.031256 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6448deb0-9935-419f-9bd4-e0b223e5ae10 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:02:10.222520 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6448deb0-9935-419f-9bd4-e0b223e5ae10 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:02:10.264722 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24611e19-2c06-4857-8027-a8f05dcde011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:02:10.265383 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24611e19-2c06-4857-8027-a8f05dcde011 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:02:10.265691 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24611e19-2c06-4857-8027-a8f05dcde011 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:02:10.265973 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24611e19-2c06-4857-8027-a8f05dcde011 None admin] PATH_INFO: `/v3/projects/5b4a260355444065be7ad89fe4339976` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:02:10.267225 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24611e19-2c06-4857-8027-a8f05dcde011 None admin] RBAC: Authorizing `identity:get_project(project_id=5b4a260355444065be7ad89fe4339976)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:02:10.268966 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24611e19-2c06-4857-8027-a8f05dcde011 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:02:10.270778 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 615/1199] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 6 20:02:10 2022] GET /identity/v3/projects/5b4a260355444065be7ad89fe4339976 => generated 361 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:02:10.277934 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0ed9fa-19c0-4c51-9345-5b1630179e43 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:02:10.374196 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:02:10.406601 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0ed9fa-19c0-4c51-9345-5b1630179e43 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:02:10.454418 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c736f12-cd8b-4965-9eb8-adef7489e89d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:02:10.455054 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c736f12-cd8b-4965-9eb8-adef7489e89d None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:02:10.455337 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c736f12-cd8b-4965-9eb8-adef7489e89d None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:02:10.455613 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c736f12-cd8b-4965-9eb8-adef7489e89d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:02:10.456395 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c736f12-cd8b-4965-9eb8-adef7489e89d None admin] RBAC: Adding query filter params (role.id=e0ab41d8ae22474688b5361bcc75e0f0, scope.project.id=5b4a260355444065be7ad89fe4339976, user.id=962de5f937b0457a8e79f0f91cb3e40c) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:02:10.456690 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c736f12-cd8b-4965-9eb8-adef7489e89d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:02:10.458258 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c736f12-cd8b-4965-9eb8-adef7489e89d None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:02:10.465769 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 585/1200] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Sep 6 20:02:10 2022] GET /identity/v3/role_assignments?user.id=962de5f937b0457a8e79f0f91cb3e40c&scope.project.id=5b4a260355444065be7ad89fe4339976&role.id=e0ab41d8ae22474688b5361bcc75e0f0 => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 20:02:20.259673 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=84 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:02:20.462342 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=30 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:03:16.002684 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f9eb2e-bde6-4be6-8f3e-43b5e7c8f814 None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:03:16.003595 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f9eb2e-bde6-4be6-8f3e-43b5e7c8f814 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:03:16.003595 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f9eb2e-bde6-4be6-8f3e-43b5e7c8f814 None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:03:16.006014 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 616/1201] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:03:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:03:16.018490 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beba759e-2c77-486e-96d2-142d3e709ef0 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:03:16.019248 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beba759e-2c77-486e-96d2-142d3e709ef0 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:03:16.019394 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beba759e-2c77-486e-96d2-142d3e709ef0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:03:16.058176 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-beba759e-2c77-486e-96d2-142d3e709ef0 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:03:16.060351 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-beba759e-2c77-486e-96d2-142d3e709ef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:03:16.250074 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 586/1202] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:03:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 20:03:16.596367 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f9eb2e-bde6-4be6-8f3e-43b5e7c8f814 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:03:16.803491 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f9eb2e-bde6-4be6-8f3e-43b5e7c8f814 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:03:16.855586 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c489e2da-f989-49b6-b098-b8b5e9a14b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:03:16.856759 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c489e2da-f989-49b6-b098-b8b5e9a14b90 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:03:16.857355 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c489e2da-f989-49b6-b098-b8b5e9a14b90 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:03:16.857959 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c489e2da-f989-49b6-b098-b8b5e9a14b90 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:03:16.887293 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c489e2da-f989-49b6-b098-b8b5e9a14b90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:03:16.889425 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c489e2da-f989-49b6-b098-b8b5e9a14b90 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:03:16.922488 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-c489e2da-f989-49b6-b098-b8b5e9a14b90 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 20:03:16.922488 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:03:16.922488 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:03:16.922488 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:03:16.922488 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:03:16.922488 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:03:16.922488 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:03:16.922488 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:03:16.922488 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:03:16.922488 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:03:16.922488 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:03:16.922488 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:03:16.922488 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 20:03:16.922488 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 20:03:16.922488 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 20:03:16.922488 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 20:03:16.924498 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:03:16.924498 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:03:16.924498 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:03:16.924498 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:03:16.924498 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:03:16.924498 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:03:16.924498 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:03:16.924498 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 20:03:16.924498 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:03:16.924498 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:03:16.924498 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:03:16.924498 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:03:16.924498 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:03:16.924498 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:03:16.924498 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:03:16.924498 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:03:16.924498 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 20:03:16.934836 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 20:03:16.934836 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 20:03:16.934836 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 20:03:16.934836 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 617/1203] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 20:03:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:03:16.938673 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beba759e-2c77-486e-96d2-142d3e709ef0 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:03:17.205526 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beba759e-2c77-486e-96d2-142d3e709ef0 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:03:17.274043 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:03:17.288553 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af859925-b93d-4412-8aac-1bc4ba101661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:03:17.289238 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af859925-b93d-4412-8aac-1bc4ba101661 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:03:17.289574 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af859925-b93d-4412-8aac-1bc4ba101661 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:03:17.289870 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af859925-b93d-4412-8aac-1bc4ba101661 None admin] PATH_INFO: `/v3/domains` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:03:17.290547 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af859925-b93d-4412-8aac-1bc4ba101661 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:03:17.290900 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af859925-b93d-4412-8aac-1bc4ba101661 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:03:17.292399 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af859925-b93d-4412-8aac-1bc4ba101661 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:03:17.317864 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 587/1204] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 6 20:03:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 379 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:03:17.325279 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c489e2da-f989-49b6-b098-b8b5e9a14b90 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:03:17.454457 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:03:17.548420 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c489e2da-f989-49b6-b098-b8b5e9a14b90 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:03:17.602763 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905e0566-b8ee-4705-a0ef-6896d3ab9d56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:03:17.604455 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905e0566-b8ee-4705-a0ef-6896d3ab9d56 None admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:03:17.605200 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905e0566-b8ee-4705-a0ef-6896d3ab9d56 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:03:17.606107 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905e0566-b8ee-4705-a0ef-6896d3ab9d56 None admin] PATH_INFO: `/v3/users` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:03:17.607641 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905e0566-b8ee-4705-a0ef-6896d3ab9d56 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:03:17.610864 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905e0566-b8ee-4705-a0ef-6896d3ab9d56 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:03:17.666140 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 618/1205] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Sep 6 20:03:17 2022] POST /identity/v3/users => generated 251 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 20:03:20.747924 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a1c435-e788-43fd-ba1b-4e85a27814a7 None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:03:20.747924 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a1c435-e788-43fd-ba1b-4e85a27814a7 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:03:20.747924 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a1c435-e788-43fd-ba1b-4e85a27814a7 None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:03:20.747924 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 588/1206] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:03:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 20:03:20.757742 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004d0acc-2d89-4a30-b42d-81a0daf72a01 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:03:20.758506 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004d0acc-2d89-4a30-b42d-81a0daf72a01 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:03:20.758506 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004d0acc-2d89-4a30-b42d-81a0daf72a01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:03:20.788999 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-004d0acc-2d89-4a30-b42d-81a0daf72a01 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:03:20.790784 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-004d0acc-2d89-4a30-b42d-81a0daf72a01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:03:20.907525 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 619/1207] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:03:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 20:03:21.228547 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a1c435-e788-43fd-ba1b-4e85a27814a7 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:03:21.472741 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a1c435-e788-43fd-ba1b-4e85a27814a7 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:03:21.541516 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b685fdd-056f-4e50-9d6d-450e1b0440fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:03:21.542454 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b685fdd-056f-4e50-9d6d-450e1b0440fd None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:03:21.542925 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b685fdd-056f-4e50-9d6d-450e1b0440fd None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:03:21.543370 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b685fdd-056f-4e50-9d6d-450e1b0440fd None admin] PATH_INFO: `/v3/roles/service` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:03:21.564148 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b685fdd-056f-4e50-9d6d-450e1b0440fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 20:03:21.572675 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-3b685fdd-056f-4e50-9d6d-450e1b0440fd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 20:03:21.572675 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:03:21.572675 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:03:21.572675 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:03:21.572675 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:03:21.572675 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:03:21.572675 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:03:21.572675 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:03:21.572675 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:03:21.572675 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:03:21.572675 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:03:21.572675 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:03:21.572675 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 20:03:21.572675 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 20:03:21.572675 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 20:03:21.572675 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 20:03:21.574219 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 20:03:21.574219 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 20:03:21.574219 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 20:03:21.574219 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 20:03:21.574219 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:03:21.574219 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:03:21.574219 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:03:21.574219 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:03:21.574219 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:03:21.574219 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:03:21.574219 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:03:21.574219 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 20:03:21.574219 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:03:21.574219 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:03:21.574219 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:03:21.574219 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:03:21.575238 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:03:21.575238 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:03:21.575238 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:03:21.575238 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:03:21.575238 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 20:03:21.575238 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 20:03:21.575238 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 20:03:21.575238 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 20:03:21.575238 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 20:03:21.575238 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 20:03:21.575238 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 20:03:21.575238 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:03:21.580214 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 589/1208] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Sep 6 20:03:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:03:21.582046 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004d0acc-2d89-4a30-b42d-81a0daf72a01 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:03:21.647229 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:03:21.763461 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004d0acc-2d89-4a30-b42d-81a0daf72a01 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:03:21.868762 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ddbb77-d7d5-442d-9ad1-6e2df79e491f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:03:21.870177 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ddbb77-d7d5-442d-9ad1-6e2df79e491f None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:03:21.870759 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ddbb77-d7d5-442d-9ad1-6e2df79e491f None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:03:21.871372 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ddbb77-d7d5-442d-9ad1-6e2df79e491f None admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:03:21.873000 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ddbb77-d7d5-442d-9ad1-6e2df79e491f None admin] RBAC: Adding query filter params (name=service) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:03:21.873832 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ddbb77-d7d5-442d-9ad1-6e2df79e491f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:03:21.877074 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ddbb77-d7d5-442d-9ad1-6e2df79e491f None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:03:21.900115 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 620/1209] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Sep 6 20:03:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 20:03:21.913864 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b685fdd-056f-4e50-9d6d-450e1b0440fd None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:03:22.029780 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:03:22.162552 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b685fdd-056f-4e50-9d6d-450e1b0440fd None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:03:22.244407 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d217bb-49ab-4d2f-976f-6f01b7c93256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:03:22.245585 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d217bb-49ab-4d2f-976f-6f01b7c93256 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:03:22.246182 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d217bb-49ab-4d2f-976f-6f01b7c93256 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:03:22.246653 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d217bb-49ab-4d2f-976f-6f01b7c93256 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:03:22.269648 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d217bb-49ab-4d2f-976f-6f01b7c93256 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:03:22.272768 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d217bb-49ab-4d2f-976f-6f01b7c93256 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:03:22.307303 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-57d217bb-49ab-4d2f-976f-6f01b7c93256 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 20:03:22.307303 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:03:22.307303 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:03:22.307303 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:03:22.307303 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:03:22.307303 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:03:22.307303 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:03:22.307303 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:03:22.307303 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:03:22.307303 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:03:22.307303 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:03:22.307303 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:03:22.307303 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 20:03:22.307303 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 20:03:22.307303 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 20:03:22.307303 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 20:03:22.309458 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:03:22.309458 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:03:22.309458 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:03:22.309458 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:03:22.309458 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:03:22.309458 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:03:22.309458 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:03:22.309458 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 20:03:22.309458 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:03:22.309458 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:03:22.309458 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:03:22.309458 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:03:22.309458 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:03:22.309458 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:03:22.309458 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:03:22.309458 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:03:22.309458 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 20:03:22.315925 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 20:03:22.315925 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 20:03:22.315925 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:03:22.315925 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 590/1210] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 20:03:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 397 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:03:22.319741 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ddbb77-d7d5-442d-9ad1-6e2df79e491f None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:03:22.523148 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:03:22.532756 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ddbb77-d7d5-442d-9ad1-6e2df79e491f None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:03:22.608679 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0718365a-876e-43e4-a756-fb618c7289f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:03:22.610059 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0718365a-876e-43e4-a756-fb618c7289f8 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:03:22.610557 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0718365a-876e-43e4-a756-fb618c7289f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:03:22.611102 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0718365a-876e-43e4-a756-fb618c7289f8 None admin] PATH_INFO: `/v3/domains` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:03:22.612662 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0718365a-876e-43e4-a756-fb618c7289f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:03:22.613231 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0718365a-876e-43e4-a756-fb618c7289f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:03:22.615976 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0718365a-876e-43e4-a756-fb618c7289f8 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:03:22.647268 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 621/1211] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 6 20:03:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 20:03:22.653156 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d217bb-49ab-4d2f-976f-6f01b7c93256 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:03:22.882136 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d217bb-49ab-4d2f-976f-6f01b7c93256 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:03:22.888235 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:03:22.957250 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3b594b-9054-4d85-a9df-34747b6f6170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:03:22.958540 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3b594b-9054-4d85-a9df-34747b6f6170 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:03:22.960893 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3b594b-9054-4d85-a9df-34747b6f6170 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:03:22.960893 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3b594b-9054-4d85-a9df-34747b6f6170 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:03:22.973120 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3b594b-9054-4d85-a9df-34747b6f6170 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:03:22.975795 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3b594b-9054-4d85-a9df-34747b6f6170 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:03:22.998936 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-0a3b594b-9054-4d85-a9df-34747b6f6170 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 06 20:03:22.998936 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:03:22.998936 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:03:22.998936 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:03:22.998936 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:03:22.998936 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:03:22.998936 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:03:22.998936 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:03:22.998936 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:03:22.998936 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:03:22.998936 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:03:22.998936 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:03:22.998936 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 20:03:22.998936 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 20:03:22.998936 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 20:03:22.998936 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 20:03:23.001144 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:03:23.001144 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:03:23.001144 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 20:03:23.001144 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 20:03:23.001144 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 20:03:23.001144 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 20:03:23.001144 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:03:23.001144 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:03:23.001144 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:03:23.001144 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:03:23.001144 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:03:23.001144 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 20:03:23.001144 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:03:23.001144 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:03:23.001144 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:03:23.001144 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:03:23.001144 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:03:23.001144 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:03:23.003314 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:03:23.003314 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:03:23.003314 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 20:03:23.003314 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 20:03:23.003314 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 20:03:23.003314 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 20:03:23.003314 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 20:03:23.003314 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 20:03:23.003314 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 06 20:03:23.003314 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:03:23.010482 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 591/1212] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 20:03:22 2022] GET /identity/v3/users/placement => generated 87 bytes in 357 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:03:23.011712 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0718365a-876e-43e4-a756-fb618c7289f8 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:03:23.191509 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0718365a-876e-43e4-a756-fb618c7289f8 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:03:23.263832 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156b3667-b7ea-4137-a38f-2c0eb2ee8d09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:03:23.264667 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156b3667-b7ea-4137-a38f-2c0eb2ee8d09 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:03:23.265128 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156b3667-b7ea-4137-a38f-2c0eb2ee8d09 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:03:23.265902 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156b3667-b7ea-4137-a38f-2c0eb2ee8d09 None admin] PATH_INFO: `/v3/users` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:03:23.266520 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156b3667-b7ea-4137-a38f-2c0eb2ee8d09 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:03:23.266725 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156b3667-b7ea-4137-a38f-2c0eb2ee8d09 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:03:23.268476 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156b3667-b7ea-4137-a38f-2c0eb2ee8d09 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:03:23.299561 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 622/1213] 10.222.0.20 () {56 vars in 1143 bytes} [Tue Sep 6 20:03:23 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:03:23.303477 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3b594b-9054-4d85-a9df-34747b6f6170 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:03:23.473430 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3b594b-9054-4d85-a9df-34747b6f6170 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:03:23.545820 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0a8d71-122d-4548-b27e-bcb401fb4a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:03:23.546933 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0a8d71-122d-4548-b27e-bcb401fb4a68 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:03:23.547814 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0a8d71-122d-4548-b27e-bcb401fb4a68 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:03:23.548536 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0a8d71-122d-4548-b27e-bcb401fb4a68 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:03:23.617027 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:03:23.701715 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0a8d71-122d-4548-b27e-bcb401fb4a68 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:03:23.704649 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0a8d71-122d-4548-b27e-bcb401fb4a68 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:03:23.944685 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 592/1214] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 6 20:03:23 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 644 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 20:03:23.954091 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156b3667-b7ea-4137-a38f-2c0eb2ee8d09 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:03:23.984972 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:03:24.126690 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156b3667-b7ea-4137-a38f-2c0eb2ee8d09 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:03:24.183871 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233b3354-4a81-40ca-aa2b-09e4f662c9dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:03:24.184533 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233b3354-4a81-40ca-aa2b-09e4f662c9dd None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:03:24.184962 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233b3354-4a81-40ca-aa2b-09e4f662c9dd None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:03:24.185184 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233b3354-4a81-40ca-aa2b-09e4f662c9dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:03:24.205674 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233b3354-4a81-40ca-aa2b-09e4f662c9dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:03:24.209350 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233b3354-4a81-40ca-aa2b-09e4f662c9dd None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:03:24.239740 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-233b3354-4a81-40ca-aa2b-09e4f662c9dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 20:03:24.239740 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:03:24.239740 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:03:24.239740 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:03:24.239740 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:03:24.239740 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:03:24.239740 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:03:24.239740 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:03:24.239740 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:03:24.239740 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:03:24.239740 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:03:24.239740 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:03:24.239740 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 20:03:24.239740 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 20:03:24.239740 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 20:03:24.239740 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 20:03:24.240891 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:03:24.240891 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:03:24.240891 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:03:24.240891 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:03:24.240891 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:03:24.240891 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:03:24.240891 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:03:24.240891 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 20:03:24.240891 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:03:24.240891 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:03:24.240891 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:03:24.240891 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:03:24.240891 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:03:24.240891 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:03:24.240891 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:03:24.240891 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:03:24.240891 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 20:03:24.241835 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 20:03:24.241835 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 20:03:24.241835 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 20:03:24.242410 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 623/1215] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 20:03:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:03:24.251529 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0a8d71-122d-4548-b27e-bcb401fb4a68 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:03:24.374247 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:03:24.445821 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0a8d71-122d-4548-b27e-bcb401fb4a68 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:03:24.520632 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8088ccba-f5fa-4f9f-8632-3571770f8ce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:03:24.521796 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8088ccba-f5fa-4f9f-8632-3571770f8ce9 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:03:24.522175 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8088ccba-f5fa-4f9f-8632-3571770f8ce9 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:03:24.522468 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8088ccba-f5fa-4f9f-8632-3571770f8ce9 None admin] PATH_INFO: `/v3/domains` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:03:24.523178 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8088ccba-f5fa-4f9f-8632-3571770f8ce9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:03:24.523574 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8088ccba-f5fa-4f9f-8632-3571770f8ce9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:03:24.524938 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8088ccba-f5fa-4f9f-8632-3571770f8ce9 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:03:24.541333 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 593/1216] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 6 20:03:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:03:24.547930 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233b3354-4a81-40ca-aa2b-09e4f662c9dd None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:03:24.680222 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:03:24.741663 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233b3354-4a81-40ca-aa2b-09e4f662c9dd None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:03:24.814082 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7be915d-a8be-4fca-a548-2db27a298096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:03:24.815123 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7be915d-a8be-4fca-a548-2db27a298096 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:03:24.815624 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7be915d-a8be-4fca-a548-2db27a298096 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:03:24.816100 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7be915d-a8be-4fca-a548-2db27a298096 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:03:24.829758 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7be915d-a8be-4fca-a548-2db27a298096 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:03:24.832253 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7be915d-a8be-4fca-a548-2db27a298096 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:03:24.852760 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-c7be915d-a8be-4fca-a548-2db27a298096 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 20:03:24.852760 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:03:24.852760 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:03:24.852760 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:03:24.852760 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:03:24.852760 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:03:24.852760 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:03:24.852760 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:03:24.852760 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:03:24.852760 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:03:24.852760 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:03:24.852760 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:03:24.852760 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 20:03:24.852760 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 20:03:24.852760 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 20:03:24.855506 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 20:03:24.855506 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:03:24.855506 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:03:24.855506 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:03:24.855506 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:03:24.855506 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:03:24.855506 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:03:24.855506 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:03:24.855506 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 20:03:24.855506 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:03:24.855506 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:03:24.855506 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:03:24.855506 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:03:24.855506 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:03:24.855506 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:03:24.855506 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:03:24.855506 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:03:24.857375 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 20:03:24.857375 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 20:03:24.857375 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 20:03:24.857375 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 20:03:24.857375 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 20:03:24.857375 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 20:03:24.857375 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 20:03:24.857375 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 20:03:24.857375 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 624/1217] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 6 20:03:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:03:24.863550 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8088ccba-f5fa-4f9f-8632-3571770f8ce9 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:03:25.028763 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8088ccba-f5fa-4f9f-8632-3571770f8ce9 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:03:25.058851 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:03:25.078910 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612b285d-a19a-4ddb-b2c5-37afe36e61bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:03:25.079727 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612b285d-a19a-4ddb-b2c5-37afe36e61bd None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:03:25.080235 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612b285d-a19a-4ddb-b2c5-37afe36e61bd None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:03:25.080602 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612b285d-a19a-4ddb-b2c5-37afe36e61bd None admin] PATH_INFO: `/v3/projects` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:03:25.081645 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612b285d-a19a-4ddb-b2c5-37afe36e61bd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:03:25.082143 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612b285d-a19a-4ddb-b2c5-37afe36e61bd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:03:25.084460 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612b285d-a19a-4ddb-b2c5-37afe36e61bd None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:03:25.104756 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 594/1218] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Sep 6 20:03:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:03:25.113535 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7be915d-a8be-4fca-a548-2db27a298096 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:03:25.259551 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7be915d-a8be-4fca-a548-2db27a298096 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:03:25.300617 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:03:25.309923 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7500bb71-df91-465b-ba8f-5e575a8200d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:03:25.310582 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7500bb71-df91-465b-ba8f-5e575a8200d5 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:03:25.310866 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7500bb71-df91-465b-ba8f-5e575a8200d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:03:25.311183 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7500bb71-df91-465b-ba8f-5e575a8200d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:03:25.311925 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7500bb71-df91-465b-ba8f-5e575a8200d5 None admin] RBAC: Adding query filter params (role.id=b950bf73c93e4f7ebed923ff14e39faa, scope.project.id=6aef43493bb1420a9e122313d8db7d5a, user.id=f57872de7bb74ea48d30e93e51830aba) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:03:25.312239 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7500bb71-df91-465b-ba8f-5e575a8200d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:03:25.313791 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7500bb71-df91-465b-ba8f-5e575a8200d5 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:03:25.321128 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 625/1219] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Sep 6 20:03:25 2022] GET /identity/v3/role_assignments?user.id=f57872de7bb74ea48d30e93e51830aba&scope.project.id=6aef43493bb1420a9e122313d8db7d5a&role.id=b950bf73c93e4f7ebed923ff14e39faa => generated 259 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:03:27.843449 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce336fb-45d6-4e87-91c9-09434f40c295 None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:03:27.844337 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce336fb-45d6-4e87-91c9-09434f40c295 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:03:27.844489 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce336fb-45d6-4e87-91c9-09434f40c295 None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:03:27.846953 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 595/1220] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:03:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:03:27.856142 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7addf3c-b62b-4dac-849c-9ebcf65cc7ca None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:03:27.856970 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7addf3c-b62b-4dac-849c-9ebcf65cc7ca None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:03:27.857099 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7addf3c-b62b-4dac-849c-9ebcf65cc7ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:03:27.887305 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-d7addf3c-b62b-4dac-849c-9ebcf65cc7ca None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:03:27.888963 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-d7addf3c-b62b-4dac-849c-9ebcf65cc7ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:03:28.028857 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 626/1221] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:03:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 20:03:28.404162 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce336fb-45d6-4e87-91c9-09434f40c295 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:03:28.540157 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce336fb-45d6-4e87-91c9-09434f40c295 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:03:28.582851 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999597c4-9d80-4b38-9304-fda5fa6081b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:03:28.583825 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999597c4-9d80-4b38-9304-fda5fa6081b5 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:03:28.584121 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999597c4-9d80-4b38-9304-fda5fa6081b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:03:28.584924 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999597c4-9d80-4b38-9304-fda5fa6081b5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:03:28.602599 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999597c4-9d80-4b38-9304-fda5fa6081b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 20:03:28.611405 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-999597c4-9d80-4b38-9304-fda5fa6081b5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 20:03:28.611405 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:03:28.611405 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:03:28.611405 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:03:28.611405 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:03:28.611405 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:03:28.611405 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:03:28.611405 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:03:28.611405 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:03:28.611405 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:03:28.611405 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:03:28.611405 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:03:28.611405 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 20:03:28.611405 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 20:03:28.611405 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 20:03:28.611405 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 20:03:28.613547 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 20:03:28.613547 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 20:03:28.613547 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 20:03:28.613547 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 20:03:28.613547 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:03:28.613547 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:03:28.613547 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:03:28.613547 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:03:28.613547 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:03:28.613547 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:03:28.613547 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:03:28.613547 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 20:03:28.613547 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:03:28.613547 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:03:28.613547 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:03:28.613547 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:03:28.617446 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:03:28.617446 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:03:28.617446 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:03:28.617446 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:03:28.617446 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 20:03:28.617446 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 20:03:28.617446 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 20:03:28.617446 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 20:03:28.617446 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 20:03:28.617446 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 20:03:28.617446 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 20:03:28.617446 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:03:28.617446 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 596/1222] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Sep 6 20:03:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:03:28.621468 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7addf3c-b62b-4dac-849c-9ebcf65cc7ca None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:03:28.808751 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7addf3c-b62b-4dac-849c-9ebcf65cc7ca None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:03:28.858446 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:03:28.884107 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc576fb-2091-41bc-a31a-a4c90cb58180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:03:28.885182 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc576fb-2091-41bc-a31a-a4c90cb58180 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:03:28.885708 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc576fb-2091-41bc-a31a-a4c90cb58180 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:03:28.886374 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc576fb-2091-41bc-a31a-a4c90cb58180 None admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:03:28.888192 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc576fb-2091-41bc-a31a-a4c90cb58180 None admin] RBAC: Adding query filter params (name=service) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:03:28.888898 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc576fb-2091-41bc-a31a-a4c90cb58180 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:03:28.892919 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc576fb-2091-41bc-a31a-a4c90cb58180 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:03:28.914755 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 627/1223] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Sep 6 20:03:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 20:03:28.924583 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999597c4-9d80-4b38-9304-fda5fa6081b5 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:03:28.993840 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:03:29.103708 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999597c4-9d80-4b38-9304-fda5fa6081b5 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:03:29.170600 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc8db22-fe0a-4ee3-91f6-8729aa3a04d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:03:29.171613 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc8db22-fe0a-4ee3-91f6-8729aa3a04d6 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:03:29.172083 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc8db22-fe0a-4ee3-91f6-8729aa3a04d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:03:29.172661 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc8db22-fe0a-4ee3-91f6-8729aa3a04d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:03:29.194432 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc8db22-fe0a-4ee3-91f6-8729aa3a04d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:03:29.197947 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc8db22-fe0a-4ee3-91f6-8729aa3a04d6 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:03:29.233121 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-afc8db22-fe0a-4ee3-91f6-8729aa3a04d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 20:03:29.233121 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:03:29.233121 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:03:29.233121 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:03:29.233121 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:03:29.233121 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:03:29.233121 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:03:29.233121 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:03:29.233121 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:03:29.233121 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:03:29.233121 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:03:29.233121 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:03:29.233121 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 20:03:29.233121 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 20:03:29.233121 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 20:03:29.233121 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 20:03:29.234630 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:03:29.234630 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:03:29.234630 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:03:29.234630 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:03:29.234630 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:03:29.234630 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:03:29.234630 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:03:29.234630 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 20:03:29.234630 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:03:29.234630 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:03:29.234630 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:03:29.234630 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:03:29.234630 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:03:29.234630 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:03:29.234630 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:03:29.234630 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:03:29.234630 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 20:03:29.235646 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 20:03:29.235646 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 20:03:29.235646 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:03:29.235646 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 597/1224] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 20:03:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:03:29.241919 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc576fb-2091-41bc-a31a-a4c90cb58180 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:03:29.408328 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc576fb-2091-41bc-a31a-a4c90cb58180 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:03:29.465063 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d6650f-e530-4989-84bc-40382152ad29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:03:29.465727 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d6650f-e530-4989-84bc-40382152ad29 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:03:29.466051 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d6650f-e530-4989-84bc-40382152ad29 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:03:29.466324 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d6650f-e530-4989-84bc-40382152ad29 None admin] PATH_INFO: `/v3/domains` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:03:29.467005 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d6650f-e530-4989-84bc-40382152ad29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:03:29.467342 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d6650f-e530-4989-84bc-40382152ad29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:03:29.469370 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d6650f-e530-4989-84bc-40382152ad29 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:03:29.488846 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 628/1225] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 6 20:03:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 20:03:29.495737 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc8db22-fe0a-4ee3-91f6-8729aa3a04d6 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:03:29.650063 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:03:29.670317 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc8db22-fe0a-4ee3-91f6-8729aa3a04d6 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:03:29.722863 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c39ef9-cf92-46c2-82a6-639cc3d559b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:03:29.723515 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c39ef9-cf92-46c2-82a6-639cc3d559b0 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:03:29.723790 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c39ef9-cf92-46c2-82a6-639cc3d559b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:03:29.724063 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c39ef9-cf92-46c2-82a6-639cc3d559b0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:03:29.734906 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c39ef9-cf92-46c2-82a6-639cc3d559b0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:03:29.736427 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c39ef9-cf92-46c2-82a6-639cc3d559b0 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:03:29.753607 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-f9c39ef9-cf92-46c2-82a6-639cc3d559b0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 06 20:03:29.753607 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:03:29.753607 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:03:29.753607 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:03:29.753607 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:03:29.753607 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:03:29.753607 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:03:29.753607 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:03:29.753607 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:03:29.753607 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:03:29.753607 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:03:29.753607 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:03:29.753607 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 20:03:29.753607 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 20:03:29.753607 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 20:03:29.753607 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 20:03:29.764378 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:03:29.764378 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:03:29.764378 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 20:03:29.764378 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 20:03:29.764378 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 20:03:29.764378 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 20:03:29.764378 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:03:29.764378 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:03:29.764378 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:03:29.764378 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:03:29.764378 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:03:29.764378 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 20:03:29.764378 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:03:29.764378 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:03:29.764378 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:03:29.764378 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:03:29.764378 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:03:29.764378 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:03:29.766551 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:03:29.766551 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:03:29.766551 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 20:03:29.766551 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 20:03:29.766551 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 20:03:29.766551 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 20:03:29.766551 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 20:03:29.766551 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 20:03:29.766551 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 06 20:03:29.766551 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:03:29.766551 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 598/1226] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 20:03:29 2022] GET /identity/v3/users/placement => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:03:29.767757 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d6650f-e530-4989-84bc-40382152ad29 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:03:29.849301 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:03:29.941134 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d6650f-e530-4989-84bc-40382152ad29 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:03:30.003862 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06afc494-ce90-49df-b97c-0ee569b17ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:03:30.005274 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06afc494-ce90-49df-b97c-0ee569b17ee2 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:03:30.006625 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06afc494-ce90-49df-b97c-0ee569b17ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:03:30.007158 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06afc494-ce90-49df-b97c-0ee569b17ee2 None admin] PATH_INFO: `/v3/users` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:03:30.009344 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06afc494-ce90-49df-b97c-0ee569b17ee2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:03:30.009991 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06afc494-ce90-49df-b97c-0ee569b17ee2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:03:30.014294 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06afc494-ce90-49df-b97c-0ee569b17ee2 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:03:30.057285 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 629/1227] 10.222.0.20 () {56 vars in 1143 bytes} [Tue Sep 6 20:03:29 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 20:03:30.065372 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c39ef9-cf92-46c2-82a6-639cc3d559b0 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:03:30.262670 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c39ef9-cf92-46c2-82a6-639cc3d559b0 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:03:30.327527 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fc56e3-c29b-471a-85c7-72946c43759f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:03:30.328152 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fc56e3-c29b-471a-85c7-72946c43759f None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:03:30.328436 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fc56e3-c29b-471a-85c7-72946c43759f None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:03:30.328711 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fc56e3-c29b-471a-85c7-72946c43759f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:03:30.344248 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fc56e3-c29b-471a-85c7-72946c43759f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:03:30.345919 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fc56e3-c29b-471a-85c7-72946c43759f None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:03:30.363974 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-41fc56e3-c29b-471a-85c7-72946c43759f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 20:03:30.363974 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:03:30.363974 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:03:30.363974 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:03:30.363974 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:03:30.363974 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:03:30.363974 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:03:30.363974 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:03:30.363974 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:03:30.363974 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:03:30.363974 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:03:30.363974 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:03:30.363974 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 20:03:30.363974 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 20:03:30.363974 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 20:03:30.363974 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 20:03:30.366517 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:03:30.366517 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:03:30.366517 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:03:30.366517 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:03:30.366517 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:03:30.366517 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:03:30.366517 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:03:30.366517 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 20:03:30.366517 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:03:30.366517 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:03:30.366517 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:03:30.366517 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:03:30.366517 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:03:30.366517 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:03:30.366517 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:03:30.366517 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:03:30.366517 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 20:03:30.368704 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 20:03:30.368704 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 20:03:30.368704 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:03:30.373063 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 599/1228] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 20:03:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:03:30.373343 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06afc494-ce90-49df-b97c-0ee569b17ee2 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:03:30.541870 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06afc494-ce90-49df-b97c-0ee569b17ee2 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:03:30.545433 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:03:30.593676 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d71602b-d1c6-4a9c-9f65-52de836b13fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:03:30.594365 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d71602b-d1c6-4a9c-9f65-52de836b13fa None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:03:30.594751 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d71602b-d1c6-4a9c-9f65-52de836b13fa None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:03:30.595146 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d71602b-d1c6-4a9c-9f65-52de836b13fa None admin] PATH_INFO: `/v3/domains` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:03:30.595913 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d71602b-d1c6-4a9c-9f65-52de836b13fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:03:30.596227 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d71602b-d1c6-4a9c-9f65-52de836b13fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:03:30.598037 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d71602b-d1c6-4a9c-9f65-52de836b13fa None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:03:30.614507 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 630/1229] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 6 20:03:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:03:30.622226 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fc56e3-c29b-471a-85c7-72946c43759f None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:03:30.638736 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:03:30.834174 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fc56e3-c29b-471a-85c7-72946c43759f None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:03:30.937838 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccbe7d1-6bf8-461a-8099-a5ea4d16d553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:03:30.938682 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccbe7d1-6bf8-461a-8099-a5ea4d16d553 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:03:30.939107 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccbe7d1-6bf8-461a-8099-a5ea4d16d553 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:03:30.939549 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccbe7d1-6bf8-461a-8099-a5ea4d16d553 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:03:30.951838 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccbe7d1-6bf8-461a-8099-a5ea4d16d553 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:03:30.954337 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccbe7d1-6bf8-461a-8099-a5ea4d16d553 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:03:30.974868 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-0ccbe7d1-6bf8-461a-8099-a5ea4d16d553 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 20:03:30.974868 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:03:30.974868 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:03:30.974868 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:03:30.974868 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:03:30.974868 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:03:30.974868 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:03:30.974868 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:03:30.974868 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:03:30.974868 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:03:30.974868 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:03:30.974868 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:03:30.974868 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 20:03:30.974868 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 20:03:30.974868 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 20:03:30.977426 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 20:03:30.977426 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:03:30.977426 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:03:30.977426 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:03:30.977426 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:03:30.977426 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:03:30.977426 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:03:30.977426 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:03:30.977426 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 20:03:30.977426 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:03:30.977426 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:03:30.977426 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:03:30.977426 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:03:30.977426 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:03:30.977426 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:03:30.977426 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:03:30.977426 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:03:30.979544 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 20:03:30.979544 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 20:03:30.979544 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 20:03:30.979544 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 20:03:30.979544 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 20:03:30.979544 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 20:03:30.979544 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 20:03:30.979544 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:03:30.979544 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 600/1230] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 6 20:03:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 356 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:03:30.986354 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d71602b-d1c6-4a9c-9f65-52de836b13fa None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:03:31.143042 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d71602b-d1c6-4a9c-9f65-52de836b13fa None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:03:31.189463 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8f0a47-4b15-4aec-b2f3-9e9a2f9780fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:03:31.190726 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8f0a47-4b15-4aec-b2f3-9e9a2f9780fb None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:03:31.191325 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8f0a47-4b15-4aec-b2f3-9e9a2f9780fb None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:03:31.191866 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8f0a47-4b15-4aec-b2f3-9e9a2f9780fb None admin] PATH_INFO: `/v3/projects` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:03:31.193292 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8f0a47-4b15-4aec-b2f3-9e9a2f9780fb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:03:31.193873 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8f0a47-4b15-4aec-b2f3-9e9a2f9780fb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:03:31.196623 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8f0a47-4b15-4aec-b2f3-9e9a2f9780fb None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:03:31.221376 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 631/1231] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Sep 6 20:03:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 20:03:31.229476 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccbe7d1-6bf8-461a-8099-a5ea4d16d553 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:03:31.320121 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:03:31.372099 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccbe7d1-6bf8-461a-8099-a5ea4d16d553 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:03:31.415329 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f35d3a-1722-4889-b43f-928eea1041a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:03:31.416193 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f35d3a-1722-4889-b43f-928eea1041a3 None admin] REQUEST_METHOD: `PUT` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:03:31.416607 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f35d3a-1722-4889-b43f-928eea1041a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:03:31.417043 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f35d3a-1722-4889-b43f-928eea1041a3 None admin] PATH_INFO: `/v3/projects/6aef43493bb1420a9e122313d8db7d5a/users/f57872de7bb74ea48d30e93e51830aba/roles/b950bf73c93e4f7ebed923ff14e39faa` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:03:31.444925 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f35d3a-1722-4889-b43f-928eea1041a3 None admin] RBAC: Authorizing `identity:create_grant(project_id=6aef43493bb1420a9e122313d8db7d5a, user_id=f57872de7bb74ea48d30e93e51830aba, role_id=b950bf73c93e4f7ebed923ff14e39faa)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:03:31.447233 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f35d3a-1722-4889-b43f-928eea1041a3 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:03:31.466297 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 601/1232] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Sep 6 20:03:31 2022] PUT /identity/v3/projects/6aef43493bb1420a9e122313d8db7d5a/users/f57872de7bb74ea48d30e93e51830aba/roles/b950bf73c93e4f7ebed923ff14e39faa => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:03:34.034653 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acce449-0880-4a7c-a547-364162dac108 None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:03:34.035282 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acce449-0880-4a7c-a547-364162dac108 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:03:34.035591 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acce449-0880-4a7c-a547-364162dac108 None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:03:34.038769 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 632/1233] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:03:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:03:34.045482 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d62ea2-6c18-4d0e-8e9c-73ef9c8709ac None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:03:34.046354 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d62ea2-6c18-4d0e-8e9c-73ef9c8709ac None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:03:34.046511 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d62ea2-6c18-4d0e-8e9c-73ef9c8709ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:03:34.070163 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-f4d62ea2-6c18-4d0e-8e9c-73ef9c8709ac None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:03:34.071541 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-f4d62ea2-6c18-4d0e-8e9c-73ef9c8709ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:03:34.186310 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 602/1234] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:03:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 20:03:34.474462 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acce449-0880-4a7c-a547-364162dac108 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:03:34.515515 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:03:34.647653 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acce449-0880-4a7c-a547-364162dac108 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:03:34.713428 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e95a40c-6233-4754-8918-28f09bec46f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:03:34.714636 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e95a40c-6233-4754-8918-28f09bec46f0 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:03:34.716110 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e95a40c-6233-4754-8918-28f09bec46f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:03:34.716488 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e95a40c-6233-4754-8918-28f09bec46f0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:03:34.731363 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e95a40c-6233-4754-8918-28f09bec46f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 20:03:34.737251 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-9e95a40c-6233-4754-8918-28f09bec46f0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 20:03:34.737251 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:03:34.737251 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:03:34.737251 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:03:34.737251 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:03:34.737251 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:03:34.737251 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:03:34.737251 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:03:34.737251 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:03:34.737251 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:03:34.737251 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:03:34.737251 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:03:34.737251 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 20:03:34.737251 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 20:03:34.737251 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 20:03:34.737251 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 20:03:34.739348 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 20:03:34.739348 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 20:03:34.739348 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 20:03:34.739348 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 20:03:34.739348 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:03:34.739348 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:03:34.739348 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:03:34.739348 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:03:34.739348 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:03:34.739348 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:03:34.739348 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:03:34.739348 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 20:03:34.739348 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:03:34.739348 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:03:34.739348 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:03:34.739348 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:03:34.741153 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:03:34.741153 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:03:34.741153 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:03:34.741153 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:03:34.741153 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 20:03:34.741153 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 20:03:34.741153 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 20:03:34.741153 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 20:03:34.741153 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 20:03:34.741153 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 20:03:34.741153 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 20:03:34.741153 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 20:03:34.741153 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 633/1235] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Sep 6 20:03:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:03:34.748470 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d62ea2-6c18-4d0e-8e9c-73ef9c8709ac None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:03:34.821065 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:03:34.924288 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d62ea2-6c18-4d0e-8e9c-73ef9c8709ac None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:03:34.991769 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a61079-3cbd-4b50-888a-3cb07e160083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:03:34.992751 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a61079-3cbd-4b50-888a-3cb07e160083 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:03:34.993386 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a61079-3cbd-4b50-888a-3cb07e160083 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:03:34.994118 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a61079-3cbd-4b50-888a-3cb07e160083 None admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:03:34.995533 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a61079-3cbd-4b50-888a-3cb07e160083 None admin] RBAC: Adding query filter params (name=service) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:03:34.996283 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a61079-3cbd-4b50-888a-3cb07e160083 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:03:34.999008 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a61079-3cbd-4b50-888a-3cb07e160083 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:03:35.016908 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 603/1236] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Sep 6 20:03:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:03:35.027026 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e95a40c-6233-4754-8918-28f09bec46f0 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:03:35.169090 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:03:35.196086 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e95a40c-6233-4754-8918-28f09bec46f0 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:03:35.259863 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d768e3-cb2b-4c9d-a4de-fd49bf2b3c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:03:35.260513 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d768e3-cb2b-4c9d-a4de-fd49bf2b3c5c None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:03:35.260902 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d768e3-cb2b-4c9d-a4de-fd49bf2b3c5c None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:03:35.261393 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d768e3-cb2b-4c9d-a4de-fd49bf2b3c5c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:03:35.278810 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d768e3-cb2b-4c9d-a4de-fd49bf2b3c5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:03:35.281094 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d768e3-cb2b-4c9d-a4de-fd49bf2b3c5c None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:03:35.305447 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-31d768e3-cb2b-4c9d-a4de-fd49bf2b3c5c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 20:03:35.305447 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:03:35.305447 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:03:35.305447 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:03:35.305447 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:03:35.305447 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:03:35.305447 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:03:35.305447 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:03:35.305447 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:03:35.305447 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:03:35.305447 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:03:35.305447 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:03:35.305447 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 20:03:35.305447 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 20:03:35.305447 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 20:03:35.305447 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 20:03:35.307664 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:03:35.307664 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:03:35.307664 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:03:35.307664 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:03:35.307664 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:03:35.307664 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:03:35.307664 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:03:35.307664 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 20:03:35.307664 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:03:35.307664 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:03:35.307664 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:03:35.307664 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:03:35.307664 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:03:35.307664 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:03:35.307664 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:03:35.307664 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:03:35.307664 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 20:03:35.318184 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 20:03:35.318184 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 20:03:35.318184 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 20:03:35.318184 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 634/1237] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 20:03:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:03:35.319272 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a61079-3cbd-4b50-888a-3cb07e160083 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:03:35.489032 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a61079-3cbd-4b50-888a-3cb07e160083 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:03:35.506665 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:03:35.565730 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6678a92-5726-4d8f-94cf-e29730f596e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:03:35.566418 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6678a92-5726-4d8f-94cf-e29730f596e9 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:03:35.566700 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6678a92-5726-4d8f-94cf-e29730f596e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:03:35.567000 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6678a92-5726-4d8f-94cf-e29730f596e9 None admin] PATH_INFO: `/v3/domains` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:03:35.567708 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6678a92-5726-4d8f-94cf-e29730f596e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:03:35.568014 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6678a92-5726-4d8f-94cf-e29730f596e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:03:35.569664 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6678a92-5726-4d8f-94cf-e29730f596e9 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:03:35.583858 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 604/1238] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 6 20:03:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:03:35.590780 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d768e3-cb2b-4c9d-a4de-fd49bf2b3c5c None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:03:35.778650 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d768e3-cb2b-4c9d-a4de-fd49bf2b3c5c None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:03:35.843494 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0956f598-ddb3-4132-a017-f43556c171d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:03:35.844205 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0956f598-ddb3-4132-a017-f43556c171d6 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:03:35.844596 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0956f598-ddb3-4132-a017-f43556c171d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:03:35.844877 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0956f598-ddb3-4132-a017-f43556c171d6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:03:35.849416 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:03:35.858873 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0956f598-ddb3-4132-a017-f43556c171d6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:03:35.861134 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0956f598-ddb3-4132-a017-f43556c171d6 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:03:35.877966 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-0956f598-ddb3-4132-a017-f43556c171d6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 06 20:03:35.877966 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:03:35.877966 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:03:35.877966 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:03:35.877966 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:03:35.877966 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:03:35.877966 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:03:35.877966 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:03:35.877966 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:03:35.877966 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:03:35.877966 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:03:35.877966 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:03:35.877966 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 20:03:35.877966 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 20:03:35.877966 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 20:03:35.877966 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 20:03:35.880054 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:03:35.880054 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:03:35.880054 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 20:03:35.880054 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 20:03:35.880054 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 20:03:35.880054 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 20:03:35.880054 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:03:35.880054 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:03:35.880054 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:03:35.880054 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:03:35.880054 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:03:35.880054 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 20:03:35.880054 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:03:35.880054 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:03:35.880054 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:03:35.880054 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:03:35.880054 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:03:35.880054 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:03:35.882116 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:03:35.882116 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:03:35.882116 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 20:03:35.882116 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 20:03:35.882116 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 20:03:35.882116 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 20:03:35.882116 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 20:03:35.882116 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 20:03:35.882116 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 06 20:03:35.882116 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 20:03:35.882116 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 635/1239] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 20:03:35 2022] GET /identity/v3/users/placement => generated 87 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:03:35.889264 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6678a92-5726-4d8f-94cf-e29730f596e9 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:03:36.095748 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6678a92-5726-4d8f-94cf-e29730f596e9 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:03:36.146229 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4aef24-4595-44a5-800e-e7ddb2d07ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:03:36.146859 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4aef24-4595-44a5-800e-e7ddb2d07ec8 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:03:36.147146 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4aef24-4595-44a5-800e-e7ddb2d07ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:03:36.147444 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4aef24-4595-44a5-800e-e7ddb2d07ec8 None admin] PATH_INFO: `/v3/users` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:03:36.148243 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4aef24-4595-44a5-800e-e7ddb2d07ec8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:03:36.148546 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4aef24-4595-44a5-800e-e7ddb2d07ec8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:03:36.150028 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4aef24-4595-44a5-800e-e7ddb2d07ec8 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:03:36.170220 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 605/1240] 10.222.0.20 () {56 vars in 1143 bytes} [Tue Sep 6 20:03:35 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:03:36.176202 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0956f598-ddb3-4132-a017-f43556c171d6 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:03:36.321178 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0956f598-ddb3-4132-a017-f43556c171d6 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:03:36.363426 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9701bd6-acf1-42ce-8d11-585fc927910d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:03:36.364394 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9701bd6-acf1-42ce-8d11-585fc927910d None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:03:36.364899 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9701bd6-acf1-42ce-8d11-585fc927910d None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:03:36.365376 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9701bd6-acf1-42ce-8d11-585fc927910d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:03:36.469094 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:03:36.479260 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9701bd6-acf1-42ce-8d11-585fc927910d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:03:36.481289 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9701bd6-acf1-42ce-8d11-585fc927910d None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:03:36.642499 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 636/1241] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 6 20:03:36 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 20:03:36.649806 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4aef24-4595-44a5-800e-e7ddb2d07ec8 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:03:36.692964 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:03:36.817321 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4aef24-4595-44a5-800e-e7ddb2d07ec8 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:03:36.875764 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e90d165-d30a-4645-955f-104c578b0382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:03:36.876533 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e90d165-d30a-4645-955f-104c578b0382 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:03:36.876977 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e90d165-d30a-4645-955f-104c578b0382 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:03:36.877684 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e90d165-d30a-4645-955f-104c578b0382 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:03:36.892946 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e90d165-d30a-4645-955f-104c578b0382 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:03:36.894629 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e90d165-d30a-4645-955f-104c578b0382 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:03:36.914919 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-0e90d165-d30a-4645-955f-104c578b0382 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 20:03:36.914919 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:03:36.914919 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:03:36.914919 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:03:36.914919 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:03:36.914919 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:03:36.914919 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:03:36.914919 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:03:36.914919 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:03:36.914919 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:03:36.914919 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:03:36.914919 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:03:36.914919 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 20:03:36.914919 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 20:03:36.914919 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 20:03:36.914919 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 20:03:36.917381 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:03:36.917381 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:03:36.917381 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:03:36.917381 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:03:36.917381 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:03:36.917381 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:03:36.917381 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:03:36.917381 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 20:03:36.917381 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:03:36.917381 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:03:36.917381 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:03:36.917381 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:03:36.917381 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:03:36.917381 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:03:36.917381 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:03:36.917381 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:03:36.917381 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 20:03:36.919813 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 20:03:36.919813 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 20:03:36.919813 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:03:36.919813 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 606/1242] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 20:03:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:03:36.926539 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9701bd6-acf1-42ce-8d11-585fc927910d None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:03:37.109066 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:03:37.120233 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9701bd6-acf1-42ce-8d11-585fc927910d None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:03:37.189935 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d853e6ad-7d1d-4ef7-a2fa-f9e74f83eec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:03:37.191126 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d853e6ad-7d1d-4ef7-a2fa-f9e74f83eec3 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:03:37.191298 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d853e6ad-7d1d-4ef7-a2fa-f9e74f83eec3 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:03:37.191784 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d853e6ad-7d1d-4ef7-a2fa-f9e74f83eec3 None admin] PATH_INFO: `/v3/domains` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:03:37.193038 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d853e6ad-7d1d-4ef7-a2fa-f9e74f83eec3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:03:37.193483 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d853e6ad-7d1d-4ef7-a2fa-f9e74f83eec3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:03:37.196270 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d853e6ad-7d1d-4ef7-a2fa-f9e74f83eec3 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:03:37.220303 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 637/1243] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 6 20:03:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:03:37.227542 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e90d165-d30a-4645-955f-104c578b0382 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:03:37.367075 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:03:37.402647 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e90d165-d30a-4645-955f-104c578b0382 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:03:37.450836 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa13464-a2e4-4281-ac63-66d7deebf323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:03:37.451526 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa13464-a2e4-4281-ac63-66d7deebf323 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:03:37.451806 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa13464-a2e4-4281-ac63-66d7deebf323 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:03:37.452081 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa13464-a2e4-4281-ac63-66d7deebf323 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:03:37.467584 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa13464-a2e4-4281-ac63-66d7deebf323 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:03:37.470359 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa13464-a2e4-4281-ac63-66d7deebf323 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:03:37.490155 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-6fa13464-a2e4-4281-ac63-66d7deebf323 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 20:03:37.490155 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:03:37.490155 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:03:37.490155 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:03:37.490155 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:03:37.490155 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:03:37.490155 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:03:37.490155 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:03:37.490155 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:03:37.490155 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:03:37.490155 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:03:37.490155 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:03:37.490155 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 20:03:37.490155 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 20:03:37.490155 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 20:03:37.492629 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 20:03:37.492629 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:03:37.492629 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:03:37.492629 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:03:37.492629 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:03:37.492629 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:03:37.492629 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:03:37.492629 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:03:37.492629 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 20:03:37.492629 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:03:37.492629 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:03:37.492629 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:03:37.492629 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:03:37.492629 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:03:37.492629 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:03:37.492629 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:03:37.492629 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:03:37.494417 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 20:03:37.494417 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 20:03:37.494417 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 20:03:37.494417 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 20:03:37.494417 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 20:03:37.494417 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 20:03:37.494417 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 20:03:37.494417 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:03:37.494417 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 607/1244] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 6 20:03:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:03:37.499479 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d853e6ad-7d1d-4ef7-a2fa-f9e74f83eec3 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:03:37.657950 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d853e6ad-7d1d-4ef7-a2fa-f9e74f83eec3 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:03:37.718497 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a5e8a6-25cf-447f-af37-93d2c8964950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:03:37.719158 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a5e8a6-25cf-447f-af37-93d2c8964950 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:03:37.719442 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a5e8a6-25cf-447f-af37-93d2c8964950 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:03:37.719731 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a5e8a6-25cf-447f-af37-93d2c8964950 None admin] PATH_INFO: `/v3/projects` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:03:37.720581 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a5e8a6-25cf-447f-af37-93d2c8964950 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:03:37.720893 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a5e8a6-25cf-447f-af37-93d2c8964950 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:03:37.722549 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a5e8a6-25cf-447f-af37-93d2c8964950 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:03:37.738046 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 638/1245] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Sep 6 20:03:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 20:03:37.746726 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa13464-a2e4-4281-ac63-66d7deebf323 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:03:37.927565 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa13464-a2e4-4281-ac63-66d7deebf323 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:03:37.989585 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27a9ecd-cdee-4ba4-bbdd-1cdc56121120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:03:37.993361 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27a9ecd-cdee-4ba4-bbdd-1cdc56121120 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:03:37.993361 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27a9ecd-cdee-4ba4-bbdd-1cdc56121120 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:03:37.993361 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27a9ecd-cdee-4ba4-bbdd-1cdc56121120 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:03:37.993932 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27a9ecd-cdee-4ba4-bbdd-1cdc56121120 None admin] RBAC: Adding query filter params (role.id=b950bf73c93e4f7ebed923ff14e39faa, scope.project.id=6aef43493bb1420a9e122313d8db7d5a, user.id=f57872de7bb74ea48d30e93e51830aba) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:03:37.994355 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27a9ecd-cdee-4ba4-bbdd-1cdc56121120 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:03:37.995996 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27a9ecd-cdee-4ba4-bbdd-1cdc56121120 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:03:38.007174 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 608/1246] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Sep 6 20:03:37 2022] GET /identity/v3/role_assignments?user.id=f57872de7bb74ea48d30e93e51830aba&scope.project.id=6aef43493bb1420a9e122313d8db7d5a&role.id=b950bf73c93e4f7ebed923ff14e39faa => generated 608 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:03:41.158941 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfd8db0-7443-4acb-8694-76825b30af43 None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:03:41.159886 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfd8db0-7443-4acb-8694-76825b30af43 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:03:41.160190 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfd8db0-7443-4acb-8694-76825b30af43 None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:03:41.163001 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 639/1247] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:03:41 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 20:03:41.173679 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b29e03-7d96-4f65-8ceb-38eccc8245c6 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:03:41.173679 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b29e03-7d96-4f65-8ceb-38eccc8245c6 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:03:41.173679 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b29e03-7d96-4f65-8ceb-38eccc8245c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:03:41.181530 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:03:41.200746 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-66b29e03-7d96-4f65-8ceb-38eccc8245c6 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:03:41.202326 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-66b29e03-7d96-4f65-8ceb-38eccc8245c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:03:41.318468 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 609/1248] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:03:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 20:03:41.671565 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfd8db0-7443-4acb-8694-76825b30af43 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:03:41.705813 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:03:41.850377 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfd8db0-7443-4acb-8694-76825b30af43 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:03:41.911643 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d596453a-cdcf-4d85-b2fb-9cc72e14a39d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:03:41.912707 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d596453a-cdcf-4d85-b2fb-9cc72e14a39d None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:03:41.913063 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d596453a-cdcf-4d85-b2fb-9cc72e14a39d None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:03:41.913460 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d596453a-cdcf-4d85-b2fb-9cc72e14a39d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:03:41.933976 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d596453a-cdcf-4d85-b2fb-9cc72e14a39d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 20:03:41.939549 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-d596453a-cdcf-4d85-b2fb-9cc72e14a39d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 20:03:41.939549 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:03:41.939549 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:03:41.939549 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:03:41.939549 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:03:41.939549 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:03:41.939549 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:03:41.939549 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:03:41.939549 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:03:41.939549 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:03:41.939549 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:03:41.939549 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:03:41.939549 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 20:03:41.939549 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 20:03:41.939549 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 20:03:41.939549 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 20:03:41.942302 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 20:03:41.942302 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 20:03:41.942302 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 20:03:41.942302 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 20:03:41.942302 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:03:41.942302 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:03:41.942302 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:03:41.942302 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:03:41.942302 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:03:41.942302 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:03:41.942302 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:03:41.942302 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 20:03:41.942302 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:03:41.942302 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:03:41.942302 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:03:41.942302 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:03:41.943414 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:03:41.943414 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:03:41.943414 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:03:41.943414 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:03:41.943414 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 20:03:41.943414 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 20:03:41.943414 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 20:03:41.943414 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 20:03:41.943414 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 20:03:41.943414 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 20:03:41.943414 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 20:03:41.943414 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 20:03:41.943414 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 640/1249] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Sep 6 20:03:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:03:41.951826 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b29e03-7d96-4f65-8ceb-38eccc8245c6 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:03:42.111523 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b29e03-7d96-4f65-8ceb-38eccc8245c6 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:03:42.152224 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:03:42.161297 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120bb8d8-64b8-45d0-a3b0-11a9185993d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:03:42.161953 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120bb8d8-64b8-45d0-a3b0-11a9185993d6 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:03:42.162292 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120bb8d8-64b8-45d0-a3b0-11a9185993d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:03:42.162532 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120bb8d8-64b8-45d0-a3b0-11a9185993d6 None admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:03:42.163312 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120bb8d8-64b8-45d0-a3b0-11a9185993d6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:03:42.163622 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120bb8d8-64b8-45d0-a3b0-11a9185993d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:03:42.165647 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120bb8d8-64b8-45d0-a3b0-11a9185993d6 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:03:42.181556 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 610/1250] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 6 20:03:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:03:42.185207 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d596453a-cdcf-4d85-b2fb-9cc72e14a39d None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:03:42.314286 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:03:42.360222 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d596453a-cdcf-4d85-b2fb-9cc72e14a39d None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:03:42.409468 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb553f6a-630e-473d-b632-c8e763601d27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:03:42.410161 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb553f6a-630e-473d-b632-c8e763601d27 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:03:42.410446 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb553f6a-630e-473d-b632-c8e763601d27 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:03:42.410770 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb553f6a-630e-473d-b632-c8e763601d27 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:03:42.423079 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb553f6a-630e-473d-b632-c8e763601d27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:03:42.424589 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb553f6a-630e-473d-b632-c8e763601d27 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:03:42.442779 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-eb553f6a-630e-473d-b632-c8e763601d27 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 20:03:42.442779 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:03:42.442779 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:03:42.442779 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:03:42.442779 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:03:42.442779 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:03:42.442779 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:03:42.442779 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:03:42.442779 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:03:42.442779 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:03:42.442779 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:03:42.442779 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:03:42.442779 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 20:03:42.442779 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 20:03:42.442779 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 20:03:42.442779 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 20:03:42.444557 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:03:42.444557 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:03:42.444557 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:03:42.444557 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:03:42.444557 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:03:42.444557 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:03:42.444557 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:03:42.444557 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 20:03:42.444557 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:03:42.444557 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:03:42.444557 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:03:42.444557 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:03:42.444557 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:03:42.444557 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:03:42.444557 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:03:42.444557 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:03:42.444557 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 20:03:42.446322 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 20:03:42.446322 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 20:03:42.446322 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 20:03:42.446322 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 641/1251] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 20:03:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:03:42.455592 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120bb8d8-64b8-45d0-a3b0-11a9185993d6 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:03:42.627432 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120bb8d8-64b8-45d0-a3b0-11a9185993d6 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:03:42.692452 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2cca4f-c99c-40f2-9546-0468fb3913f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:03:42.693138 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2cca4f-c99c-40f2-9546-0468fb3913f5 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:03:42.693317 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2cca4f-c99c-40f2-9546-0468fb3913f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:03:42.693595 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2cca4f-c99c-40f2-9546-0468fb3913f5 None admin] PATH_INFO: `/v3/domains` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:03:42.694163 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2cca4f-c99c-40f2-9546-0468fb3913f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:03:42.694682 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2cca4f-c99c-40f2-9546-0468fb3913f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:03:42.696030 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2cca4f-c99c-40f2-9546-0468fb3913f5 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:03:42.718878 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 611/1252] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 6 20:03:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:03:42.725781 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb553f6a-630e-473d-b632-c8e763601d27 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:03:42.908059 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb553f6a-630e-473d-b632-c8e763601d27 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:03:42.963597 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:03:42.976034 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035817fc-73f1-4364-9dd0-4028ba547ee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:03:42.977470 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035817fc-73f1-4364-9dd0-4028ba547ee7 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:03:42.977959 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035817fc-73f1-4364-9dd0-4028ba547ee7 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:03:42.978314 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035817fc-73f1-4364-9dd0-4028ba547ee7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:03:42.989453 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035817fc-73f1-4364-9dd0-4028ba547ee7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:03:42.991263 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035817fc-73f1-4364-9dd0-4028ba547ee7 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:03:43.007053 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-035817fc-73f1-4364-9dd0-4028ba547ee7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 06 20:03:43.007053 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:03:43.007053 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:03:43.007053 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:03:43.007053 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:03:43.007053 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:03:43.007053 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:03:43.007053 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:03:43.007053 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:03:43.007053 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:03:43.007053 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:03:43.007053 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:03:43.007053 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 20:03:43.007053 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 20:03:43.007053 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 20:03:43.007053 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 20:03:43.011447 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:03:43.011447 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:03:43.011447 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 20:03:43.011447 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 20:03:43.011447 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 20:03:43.011447 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 20:03:43.011447 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:03:43.011447 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:03:43.011447 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:03:43.011447 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:03:43.011447 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:03:43.011447 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 20:03:43.011447 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:03:43.011447 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:03:43.011447 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:03:43.011447 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:03:43.011447 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:03:43.011447 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:03:43.013316 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:03:43.013316 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:03:43.013316 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 20:03:43.013316 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 20:03:43.013316 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 20:03:43.013316 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 20:03:43.013316 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 20:03:43.013316 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 20:03:43.013316 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 06 20:03:43.013316 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 20:03:43.013316 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 642/1253] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 20:03:42 2022] GET /identity/v3/users/placement => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 20:03:43.017800 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2cca4f-c99c-40f2-9546-0468fb3913f5 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:03:43.126412 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:03:43.207331 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2cca4f-c99c-40f2-9546-0468fb3913f5 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:03:43.283409 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257ad0d5-962c-403e-8b65-59996debc011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:03:43.284030 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257ad0d5-962c-403e-8b65-59996debc011 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:03:43.284309 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257ad0d5-962c-403e-8b65-59996debc011 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:03:43.284579 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257ad0d5-962c-403e-8b65-59996debc011 None admin] PATH_INFO: `/v3/users` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:03:43.285414 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257ad0d5-962c-403e-8b65-59996debc011 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:03:43.285770 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257ad0d5-962c-403e-8b65-59996debc011 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:03:43.287390 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257ad0d5-962c-403e-8b65-59996debc011 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:03:43.310484 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 612/1254] 10.222.0.20 () {56 vars in 1143 bytes} [Tue Sep 6 20:03:43 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:03:43.317192 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035817fc-73f1-4364-9dd0-4028ba547ee7 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:03:43.469938 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035817fc-73f1-4364-9dd0-4028ba547ee7 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:03:43.517807 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e47672-59a8-4870-8ae3-32cf585d091e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:03:43.518453 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e47672-59a8-4870-8ae3-32cf585d091e None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:03:43.518741 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e47672-59a8-4870-8ae3-32cf585d091e None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:03:43.519024 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e47672-59a8-4870-8ae3-32cf585d091e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:03:43.600514 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:03:43.616921 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e47672-59a8-4870-8ae3-32cf585d091e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:03:43.618528 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e47672-59a8-4870-8ae3-32cf585d091e None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:03:43.791294 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 643/1255] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 6 20:03:43 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 20:03:43.798558 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257ad0d5-962c-403e-8b65-59996debc011 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:03:43.981876 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257ad0d5-962c-403e-8b65-59996debc011 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:03:44.002790 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:03:44.040323 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd74bd43-1435-4057-8b80-202b570aeb8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:03:44.041083 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd74bd43-1435-4057-8b80-202b570aeb8e None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:03:44.041462 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd74bd43-1435-4057-8b80-202b570aeb8e None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:03:44.041941 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd74bd43-1435-4057-8b80-202b570aeb8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:03:44.061714 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd74bd43-1435-4057-8b80-202b570aeb8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:03:44.064260 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd74bd43-1435-4057-8b80-202b570aeb8e None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:03:44.091822 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-fd74bd43-1435-4057-8b80-202b570aeb8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 20:03:44.091822 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:03:44.091822 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:03:44.091822 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:03:44.091822 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:03:44.091822 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:03:44.091822 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:03:44.091822 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:03:44.091822 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:03:44.091822 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:03:44.091822 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:03:44.091822 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:03:44.091822 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 20:03:44.091822 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 20:03:44.091822 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 20:03:44.091822 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 20:03:44.094343 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:03:44.094343 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:03:44.094343 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:03:44.094343 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:03:44.094343 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:03:44.094343 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:03:44.094343 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:03:44.094343 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 20:03:44.094343 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:03:44.094343 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:03:44.094343 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:03:44.094343 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:03:44.094343 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:03:44.094343 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:03:44.094343 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:03:44.094343 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:03:44.094343 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 20:03:44.095857 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 20:03:44.095857 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 20:03:44.095857 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:03:44.095857 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 613/1256] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 20:03:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:03:44.101010 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e47672-59a8-4870-8ae3-32cf585d091e None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:03:44.264845 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:03:44.287937 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e47672-59a8-4870-8ae3-32cf585d091e None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:03:44.342204 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a00296a-aa73-41fc-b50c-bc7763b14be4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:03:44.343276 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a00296a-aa73-41fc-b50c-bc7763b14be4 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:03:44.343607 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a00296a-aa73-41fc-b50c-bc7763b14be4 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:03:44.343968 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a00296a-aa73-41fc-b50c-bc7763b14be4 None admin] PATH_INFO: `/v3/domains` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:03:44.344730 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a00296a-aa73-41fc-b50c-bc7763b14be4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:03:44.345128 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a00296a-aa73-41fc-b50c-bc7763b14be4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:03:44.346743 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a00296a-aa73-41fc-b50c-bc7763b14be4 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:03:44.369278 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 644/1257] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 6 20:03:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:03:44.371370 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd74bd43-1435-4057-8b80-202b570aeb8e None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:03:44.555319 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd74bd43-1435-4057-8b80-202b570aeb8e None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:03:44.621099 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a406e1c9-fb99-4aa6-a90b-e15051dec07b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:03:44.622375 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a406e1c9-fb99-4aa6-a90b-e15051dec07b None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:03:44.622787 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a406e1c9-fb99-4aa6-a90b-e15051dec07b None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:03:44.623191 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a406e1c9-fb99-4aa6-a90b-e15051dec07b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:03:44.632446 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a406e1c9-fb99-4aa6-a90b-e15051dec07b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:03:44.634654 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a406e1c9-fb99-4aa6-a90b-e15051dec07b None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:03:44.650056 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-a406e1c9-fb99-4aa6-a90b-e15051dec07b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 20:03:44.650056 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:03:44.650056 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:03:44.650056 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:03:44.650056 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:03:44.650056 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:03:44.650056 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:03:44.650056 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:03:44.650056 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:03:44.650056 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:03:44.650056 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:03:44.650056 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:03:44.650056 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 20:03:44.650056 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 20:03:44.650056 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 20:03:44.652119 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 20:03:44.652119 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:03:44.652119 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:03:44.652119 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:03:44.652119 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:03:44.652119 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:03:44.652119 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:03:44.652119 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:03:44.652119 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 20:03:44.652119 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:03:44.652119 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:03:44.652119 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:03:44.652119 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:03:44.652119 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:03:44.652119 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:03:44.652119 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:03:44.652119 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:03:44.654079 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 20:03:44.654079 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 20:03:44.654079 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 20:03:44.654079 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 20:03:44.654079 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 20:03:44.654079 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 20:03:44.654079 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 20:03:44.654079 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:03:44.654079 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 614/1258] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 6 20:03:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:03:44.661655 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a00296a-aa73-41fc-b50c-bc7763b14be4 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:03:44.805972 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a00296a-aa73-41fc-b50c-bc7763b14be4 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:03:44.860485 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7b2782-7df6-4e60-bff0-6d9cba27945c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:03:44.861545 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7b2782-7df6-4e60-bff0-6d9cba27945c None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:03:44.862030 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7b2782-7df6-4e60-bff0-6d9cba27945c None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:03:44.862465 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7b2782-7df6-4e60-bff0-6d9cba27945c None admin] PATH_INFO: `/v3/projects` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:03:44.863582 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7b2782-7df6-4e60-bff0-6d9cba27945c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:03:44.864126 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7b2782-7df6-4e60-bff0-6d9cba27945c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:03:44.866912 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7b2782-7df6-4e60-bff0-6d9cba27945c None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:03:44.892733 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 645/1259] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Sep 6 20:03:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 20:03:44.900329 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a406e1c9-fb99-4aa6-a90b-e15051dec07b None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:03:44.956727 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:03:45.084769 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a406e1c9-fb99-4aa6-a90b-e15051dec07b None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:03:45.137239 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e72c3db-4858-4ac4-b0c3-9d6eed2a7803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:03:45.137932 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e72c3db-4858-4ac4-b0c3-9d6eed2a7803 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:03:45.138276 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e72c3db-4858-4ac4-b0c3-9d6eed2a7803 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:03:45.138548 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e72c3db-4858-4ac4-b0c3-9d6eed2a7803 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:03:45.139340 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e72c3db-4858-4ac4-b0c3-9d6eed2a7803 None admin] RBAC: Adding query filter params (role.id=e0ab41d8ae22474688b5361bcc75e0f0, scope.project.id=6aef43493bb1420a9e122313d8db7d5a, user.id=f57872de7bb74ea48d30e93e51830aba) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:03:45.139643 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e72c3db-4858-4ac4-b0c3-9d6eed2a7803 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:03:45.141104 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e72c3db-4858-4ac4-b0c3-9d6eed2a7803 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:03:45.148014 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 615/1260] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Sep 6 20:03:44 2022] GET /identity/v3/role_assignments?user.id=f57872de7bb74ea48d30e93e51830aba&scope.project.id=6aef43493bb1420a9e122313d8db7d5a&role.id=e0ab41d8ae22474688b5361bcc75e0f0 => generated 259 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:03:47.667971 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d15229-1628-496c-af1d-7c824bc4720b None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:03:47.668544 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d15229-1628-496c-af1d-7c824bc4720b None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:03:47.668544 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d15229-1628-496c-af1d-7c824bc4720b None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:03:47.669899 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 646/1261] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:03:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:03:47.675774 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31764c4-ef7d-431d-92b9-bdf91a681cea None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:03:47.676637 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31764c4-ef7d-431d-92b9-bdf91a681cea None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:03:47.676637 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31764c4-ef7d-431d-92b9-bdf91a681cea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:03:47.697263 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-d31764c4-ef7d-431d-92b9-bdf91a681cea None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:03:47.698546 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-d31764c4-ef7d-431d-92b9-bdf91a681cea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:03:47.773493 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 616/1262] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:03:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 20:03:48.054757 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d15229-1628-496c-af1d-7c824bc4720b None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:03:48.067010 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:03:48.173666 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d15229-1628-496c-af1d-7c824bc4720b None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:03:48.212936 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acd20ac-2853-49c2-b8c6-7f06be6c18c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:03:48.213597 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acd20ac-2853-49c2-b8c6-7f06be6c18c4 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:03:48.213916 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acd20ac-2853-49c2-b8c6-7f06be6c18c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:03:48.214192 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acd20ac-2853-49c2-b8c6-7f06be6c18c4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:03:48.226398 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acd20ac-2853-49c2-b8c6-7f06be6c18c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 20:03:48.231978 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-2acd20ac-2853-49c2-b8c6-7f06be6c18c4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 20:03:48.231978 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:03:48.231978 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:03:48.231978 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:03:48.231978 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:03:48.231978 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:03:48.231978 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:03:48.231978 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:03:48.231978 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:03:48.231978 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:03:48.231978 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:03:48.231978 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:03:48.231978 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 20:03:48.231978 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 20:03:48.231978 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 20:03:48.231978 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 20:03:48.233400 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 20:03:48.233400 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 20:03:48.233400 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 20:03:48.233400 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 20:03:48.233400 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:03:48.233400 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:03:48.233400 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:03:48.233400 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:03:48.233400 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:03:48.233400 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:03:48.233400 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:03:48.233400 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 20:03:48.233400 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:03:48.233400 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:03:48.233400 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:03:48.233400 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:03:48.234678 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:03:48.234678 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:03:48.234678 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:03:48.234678 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:03:48.234678 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 20:03:48.234678 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 20:03:48.234678 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 20:03:48.234678 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 20:03:48.234678 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 20:03:48.234678 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 20:03:48.234678 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 20:03:48.234678 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 20:03:48.234678 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 647/1263] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Sep 6 20:03:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:03:48.242004 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31764c4-ef7d-431d-92b9-bdf91a681cea None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:03:48.264539 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:03:48.365582 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31764c4-ef7d-431d-92b9-bdf91a681cea None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:03:48.411528 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cc8d43-b899-4b4c-b6a5-66fa84a26801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:03:48.412150 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cc8d43-b899-4b4c-b6a5-66fa84a26801 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:03:48.412429 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cc8d43-b899-4b4c-b6a5-66fa84a26801 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:03:48.412710 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cc8d43-b899-4b4c-b6a5-66fa84a26801 None admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:03:48.413400 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cc8d43-b899-4b4c-b6a5-66fa84a26801 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:03:48.413751 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cc8d43-b899-4b4c-b6a5-66fa84a26801 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:03:48.415684 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cc8d43-b899-4b4c-b6a5-66fa84a26801 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:03:48.425513 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 617/1264] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 6 20:03:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:03:48.432720 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acd20ac-2853-49c2-b8c6-7f06be6c18c4 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:03:48.512240 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:03:48.551353 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acd20ac-2853-49c2-b8c6-7f06be6c18c4 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:03:48.592447 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ce27ff-ed36-4149-a76e-d8417a66164c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:03:48.593098 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ce27ff-ed36-4149-a76e-d8417a66164c None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:03:48.593386 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ce27ff-ed36-4149-a76e-d8417a66164c None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:03:48.593695 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ce27ff-ed36-4149-a76e-d8417a66164c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:03:48.606383 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ce27ff-ed36-4149-a76e-d8417a66164c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:03:48.607870 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ce27ff-ed36-4149-a76e-d8417a66164c None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:03:48.624381 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-97ce27ff-ed36-4149-a76e-d8417a66164c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 20:03:48.624381 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:03:48.624381 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:03:48.624381 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:03:48.624381 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:03:48.624381 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:03:48.624381 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:03:48.624381 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:03:48.624381 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:03:48.624381 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:03:48.624381 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:03:48.624381 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:03:48.624381 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 20:03:48.624381 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 20:03:48.624381 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 20:03:48.624381 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 20:03:48.626601 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:03:48.626601 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:03:48.626601 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:03:48.626601 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:03:48.626601 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:03:48.626601 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:03:48.626601 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:03:48.626601 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 20:03:48.626601 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:03:48.626601 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:03:48.626601 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:03:48.626601 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:03:48.626601 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:03:48.626601 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:03:48.626601 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:03:48.626601 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:03:48.626601 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 20:03:48.628293 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 20:03:48.628293 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 20:03:48.628293 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 20:03:48.628293 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 648/1265] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 20:03:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 20:03:48.631670 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cc8d43-b899-4b4c-b6a5-66fa84a26801 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:03:48.772874 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:03:48.799280 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cc8d43-b899-4b4c-b6a5-66fa84a26801 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:03:48.841284 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c31cfb-6bb0-435d-b4b8-f1d1c104df4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:03:48.841956 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c31cfb-6bb0-435d-b4b8-f1d1c104df4d None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:03:48.842250 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c31cfb-6bb0-435d-b4b8-f1d1c104df4d None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:03:48.842552 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c31cfb-6bb0-435d-b4b8-f1d1c104df4d None admin] PATH_INFO: `/v3/domains` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:03:48.843239 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c31cfb-6bb0-435d-b4b8-f1d1c104df4d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:03:48.843545 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c31cfb-6bb0-435d-b4b8-f1d1c104df4d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:03:48.844976 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c31cfb-6bb0-435d-b4b8-f1d1c104df4d None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:03:48.860748 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 618/1266] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 6 20:03:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 20:03:48.868766 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ce27ff-ed36-4149-a76e-d8417a66164c None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:03:49.007602 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ce27ff-ed36-4149-a76e-d8417a66164c None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:03:49.034082 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:03:49.053325 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a946a99-dfba-4a03-88cf-9d0bba1fcda5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:03:49.054076 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a946a99-dfba-4a03-88cf-9d0bba1fcda5 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:03:49.054354 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a946a99-dfba-4a03-88cf-9d0bba1fcda5 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:03:49.054649 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a946a99-dfba-4a03-88cf-9d0bba1fcda5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:03:49.068303 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a946a99-dfba-4a03-88cf-9d0bba1fcda5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:03:49.070622 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a946a99-dfba-4a03-88cf-9d0bba1fcda5 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:03:49.090945 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-9a946a99-dfba-4a03-88cf-9d0bba1fcda5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 06 20:03:49.090945 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:03:49.090945 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:03:49.090945 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:03:49.090945 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:03:49.090945 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:03:49.090945 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:03:49.090945 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:03:49.090945 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:03:49.090945 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:03:49.090945 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:03:49.090945 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:03:49.090945 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 20:03:49.090945 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 20:03:49.090945 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 20:03:49.090945 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 20:03:49.092830 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:03:49.092830 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:03:49.092830 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 20:03:49.092830 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 20:03:49.092830 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 20:03:49.092830 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 20:03:49.092830 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:03:49.092830 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:03:49.092830 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:03:49.092830 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:03:49.092830 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:03:49.092830 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 20:03:49.092830 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:03:49.092830 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:03:49.092830 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:03:49.092830 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:03:49.092830 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:03:49.092830 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:03:49.098583 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:03:49.098583 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:03:49.098583 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 20:03:49.098583 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 20:03:49.098583 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 20:03:49.098583 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 20:03:49.098583 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 20:03:49.098583 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 20:03:49.098583 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 06 20:03:49.098583 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 20:03:49.098583 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 649/1267] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 20:03:48 2022] GET /identity/v3/users/placement => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:03:49.101303 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c31cfb-6bb0-435d-b4b8-f1d1c104df4d None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:03:49.226315 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c31cfb-6bb0-435d-b4b8-f1d1c104df4d None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:03:49.267831 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50291bb4-c0d2-4988-90d6-e3dc9ab39ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:03:49.268488 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50291bb4-c0d2-4988-90d6-e3dc9ab39ae2 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:03:49.268887 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50291bb4-c0d2-4988-90d6-e3dc9ab39ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:03:49.269274 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50291bb4-c0d2-4988-90d6-e3dc9ab39ae2 None admin] PATH_INFO: `/v3/users` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:03:49.270207 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50291bb4-c0d2-4988-90d6-e3dc9ab39ae2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:03:49.270611 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50291bb4-c0d2-4988-90d6-e3dc9ab39ae2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:03:49.272201 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50291bb4-c0d2-4988-90d6-e3dc9ab39ae2 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:03:49.284119 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:03:49.294554 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 619/1268] 10.222.0.20 () {56 vars in 1143 bytes} [Tue Sep 6 20:03:49 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:03:49.301354 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a946a99-dfba-4a03-88cf-9d0bba1fcda5 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:03:49.423489 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a946a99-dfba-4a03-88cf-9d0bba1fcda5 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:03:49.463334 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9945491a-8121-4e86-b792-de4621180ee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:03:49.464037 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9945491a-8121-4e86-b792-de4621180ee5 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:03:49.464348 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9945491a-8121-4e86-b792-de4621180ee5 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:03:49.464641 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9945491a-8121-4e86-b792-de4621180ee5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:03:49.479380 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9945491a-8121-4e86-b792-de4621180ee5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:03:49.481990 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9945491a-8121-4e86-b792-de4621180ee5 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:03:49.499917 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-9945491a-8121-4e86-b792-de4621180ee5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 20:03:49.499917 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:03:49.499917 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:03:49.499917 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:03:49.499917 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:03:49.499917 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:03:49.499917 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:03:49.499917 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:03:49.499917 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:03:49.499917 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:03:49.499917 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:03:49.499917 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:03:49.499917 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 20:03:49.499917 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 20:03:49.499917 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 20:03:49.499917 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 20:03:49.501832 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:03:49.501832 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:03:49.501832 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:03:49.501832 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:03:49.501832 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:03:49.501832 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:03:49.501832 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:03:49.501832 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 20:03:49.501832 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:03:49.501832 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:03:49.501832 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:03:49.501832 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:03:49.501832 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:03:49.501832 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:03:49.501832 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:03:49.501832 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:03:49.501832 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 20:03:49.504162 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 20:03:49.504162 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 20:03:49.504162 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 20:03:49.504162 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 650/1269] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 20:03:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:03:49.509444 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50291bb4-c0d2-4988-90d6-e3dc9ab39ae2 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:03:49.634552 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50291bb4-c0d2-4988-90d6-e3dc9ab39ae2 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:03:49.675903 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5ee893-a167-4705-8f1b-1be8e5582985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:03:49.676643 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5ee893-a167-4705-8f1b-1be8e5582985 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:03:49.677009 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5ee893-a167-4705-8f1b-1be8e5582985 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:03:49.677346 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5ee893-a167-4705-8f1b-1be8e5582985 None admin] PATH_INFO: `/v3/domains` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:03:49.678164 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5ee893-a167-4705-8f1b-1be8e5582985 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:03:49.678530 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5ee893-a167-4705-8f1b-1be8e5582985 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:03:49.679951 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5ee893-a167-4705-8f1b-1be8e5582985 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:03:49.694230 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 620/1270] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 6 20:03:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:03:49.701127 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9945491a-8121-4e86-b792-de4621180ee5 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:03:49.737313 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:03:49.826027 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9945491a-8121-4e86-b792-de4621180ee5 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:03:49.870547 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf21ab54-8ea6-44bb-8b12-5c2111ab70db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:03:49.871945 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf21ab54-8ea6-44bb-8b12-5c2111ab70db None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:03:49.871945 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf21ab54-8ea6-44bb-8b12-5c2111ab70db None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:03:49.872448 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf21ab54-8ea6-44bb-8b12-5c2111ab70db None admin] PATH_INFO: `/v3/projects/service` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:03:49.884604 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf21ab54-8ea6-44bb-8b12-5c2111ab70db None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:03:49.888592 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf21ab54-8ea6-44bb-8b12-5c2111ab70db None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:03:49.912550 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-cf21ab54-8ea6-44bb-8b12-5c2111ab70db None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 20:03:49.912550 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:03:49.912550 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:03:49.912550 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:03:49.912550 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:03:49.912550 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:03:49.912550 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:03:49.912550 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:03:49.912550 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:03:49.912550 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:03:49.912550 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:03:49.912550 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:03:49.912550 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 20:03:49.912550 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 20:03:49.912550 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 20:03:49.914315 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 20:03:49.914315 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:03:49.914315 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:03:49.914315 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:03:49.914315 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:03:49.914315 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:03:49.914315 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:03:49.914315 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:03:49.914315 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 20:03:49.914315 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:03:49.914315 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:03:49.914315 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:03:49.914315 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:03:49.914315 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:03:49.914315 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:03:49.914315 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:03:49.914315 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:03:49.916246 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 20:03:49.916246 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 20:03:49.916246 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 20:03:49.916246 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 20:03:49.916246 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 20:03:49.916246 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 20:03:49.916246 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 20:03:49.916246 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 20:03:49.916246 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 651/1271] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 6 20:03:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 20:03:49.923588 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5ee893-a167-4705-8f1b-1be8e5582985 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:03:50.010381 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:03:50.062903 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5ee893-a167-4705-8f1b-1be8e5582985 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:03:50.106322 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5944f7ee-3e4a-4b27-b718-e6b02d50dc04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:03:50.107019 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5944f7ee-3e4a-4b27-b718-e6b02d50dc04 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:03:50.107337 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5944f7ee-3e4a-4b27-b718-e6b02d50dc04 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:03:50.107695 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5944f7ee-3e4a-4b27-b718-e6b02d50dc04 None admin] PATH_INFO: `/v3/projects` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:03:50.108460 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5944f7ee-3e4a-4b27-b718-e6b02d50dc04 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:03:50.108826 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5944f7ee-3e4a-4b27-b718-e6b02d50dc04 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:03:50.110399 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5944f7ee-3e4a-4b27-b718-e6b02d50dc04 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:03:50.125201 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 621/1272] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Sep 6 20:03:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:03:50.132955 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf21ab54-8ea6-44bb-8b12-5c2111ab70db None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:03:50.248000 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:03:50.259726 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf21ab54-8ea6-44bb-8b12-5c2111ab70db None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:03:50.297070 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353d56d4-8386-4fe4-b215-c9088b3f1cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:03:50.297783 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353d56d4-8386-4fe4-b215-c9088b3f1cd6 None admin] REQUEST_METHOD: `PUT` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:03:50.298069 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353d56d4-8386-4fe4-b215-c9088b3f1cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:03:50.298354 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353d56d4-8386-4fe4-b215-c9088b3f1cd6 None admin] PATH_INFO: `/v3/projects/6aef43493bb1420a9e122313d8db7d5a/users/f57872de7bb74ea48d30e93e51830aba/roles/e0ab41d8ae22474688b5361bcc75e0f0` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:03:50.300463 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353d56d4-8386-4fe4-b215-c9088b3f1cd6 None admin] RBAC: Authorizing `identity:create_grant(project_id=6aef43493bb1420a9e122313d8db7d5a, user_id=f57872de7bb74ea48d30e93e51830aba, role_id=e0ab41d8ae22474688b5361bcc75e0f0)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:03:50.301991 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353d56d4-8386-4fe4-b215-c9088b3f1cd6 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:03:50.321089 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 652/1273] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Sep 6 20:03:50 2022] PUT /identity/v3/projects/6aef43493bb1420a9e122313d8db7d5a/users/f57872de7bb74ea48d30e93e51830aba/roles/e0ab41d8ae22474688b5361bcc75e0f0 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:03:52.687999 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac645ab7-f192-4297-91d0-074fc8080f9e None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:03:52.688583 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac645ab7-f192-4297-91d0-074fc8080f9e None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:03:52.688583 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac645ab7-f192-4297-91d0-074fc8080f9e None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:03:52.690188 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 622/1274] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:03:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:03:52.697203 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f0eb20-b25c-4c3b-bb15-8529c5aa829e None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:03:52.697203 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f0eb20-b25c-4c3b-bb15-8529c5aa829e None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:03:52.697203 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f0eb20-b25c-4c3b-bb15-8529c5aa829e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:03:52.718706 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-83f0eb20-b25c-4c3b-bb15-8529c5aa829e None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:03:52.719951 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-83f0eb20-b25c-4c3b-bb15-8529c5aa829e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:03:52.811770 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 653/1275] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:03:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 20:03:53.110207 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac645ab7-f192-4297-91d0-074fc8080f9e None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:03:53.237521 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac645ab7-f192-4297-91d0-074fc8080f9e None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:03:53.258342 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:03:53.279002 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fafce04-690a-4438-a4fc-59294ec4c0fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:03:53.279747 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fafce04-690a-4438-a4fc-59294ec4c0fd None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:03:53.279928 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fafce04-690a-4438-a4fc-59294ec4c0fd None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:03:53.280197 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fafce04-690a-4438-a4fc-59294ec4c0fd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:03:53.292178 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fafce04-690a-4438-a4fc-59294ec4c0fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 20:03:53.297181 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-8fafce04-690a-4438-a4fc-59294ec4c0fd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 20:03:53.297181 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:03:53.297181 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:03:53.297181 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:03:53.297181 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:03:53.297181 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:03:53.297181 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:03:53.297181 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:03:53.297181 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:03:53.297181 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:03:53.297181 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:03:53.297181 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:03:53.297181 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 20:03:53.297181 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 20:03:53.297181 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 20:03:53.297181 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 20:03:53.299050 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 20:03:53.299050 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 20:03:53.299050 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 20:03:53.299050 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 20:03:53.299050 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:03:53.299050 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:03:53.299050 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:03:53.299050 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:03:53.299050 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:03:53.299050 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:03:53.299050 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:03:53.299050 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 20:03:53.299050 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:03:53.299050 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:03:53.299050 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:03:53.299050 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:03:53.300831 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:03:53.300831 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:03:53.300831 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:03:53.300831 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:03:53.300831 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 20:03:53.300831 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 20:03:53.300831 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 20:03:53.300831 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 20:03:53.300831 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 20:03:53.300831 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 20:03:53.300831 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 20:03:53.300831 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:03:53.300831 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 623/1276] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Sep 6 20:03:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 20:03:53.304905 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f0eb20-b25c-4c3b-bb15-8529c5aa829e None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:03:53.395772 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:03:53.419521 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f0eb20-b25c-4c3b-bb15-8529c5aa829e None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:03:53.457789 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbd1a94-0594-4bb1-85b7-c8eb5e041e3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:03:53.458414 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbd1a94-0594-4bb1-85b7-c8eb5e041e3d None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:03:53.458699 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbd1a94-0594-4bb1-85b7-c8eb5e041e3d None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:03:53.458967 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbd1a94-0594-4bb1-85b7-c8eb5e041e3d None admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:03:53.459654 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbd1a94-0594-4bb1-85b7-c8eb5e041e3d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:03:53.459956 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbd1a94-0594-4bb1-85b7-c8eb5e041e3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:03:53.461367 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbd1a94-0594-4bb1-85b7-c8eb5e041e3d None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:03:53.473082 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 654/1277] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 6 20:03:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:03:53.480080 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fafce04-690a-4438-a4fc-59294ec4c0fd None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:03:53.607694 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fafce04-690a-4438-a4fc-59294ec4c0fd None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:03:53.649312 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca2a82b-1da8-4adf-aebc-3f6ce74810c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:03:53.650052 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca2a82b-1da8-4adf-aebc-3f6ce74810c2 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:03:53.650338 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca2a82b-1da8-4adf-aebc-3f6ce74810c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:03:53.650629 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca2a82b-1da8-4adf-aebc-3f6ce74810c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:03:53.663538 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca2a82b-1da8-4adf-aebc-3f6ce74810c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:03:53.665518 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca2a82b-1da8-4adf-aebc-3f6ce74810c2 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:03:53.682166 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-9ca2a82b-1da8-4adf-aebc-3f6ce74810c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 20:03:53.682166 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:03:53.682166 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:03:53.682166 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:03:53.682166 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:03:53.682166 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:03:53.682166 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:03:53.682166 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:03:53.682166 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:03:53.682166 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:03:53.682166 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:03:53.682166 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:03:53.682166 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 20:03:53.682166 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 20:03:53.682166 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 20:03:53.682166 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 20:03:53.683499 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:03:53.683499 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:03:53.683499 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:03:53.683499 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:03:53.683499 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:03:53.683499 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:03:53.683499 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:03:53.683499 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 20:03:53.683499 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:03:53.683499 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:03:53.683499 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:03:53.683499 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:03:53.683499 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:03:53.683499 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:03:53.683499 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:03:53.683499 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:03:53.683499 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 20:03:53.684720 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 20:03:53.684720 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 20:03:53.684720 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:03:53.684720 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 624/1278] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 20:03:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:03:53.690135 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbd1a94-0594-4bb1-85b7-c8eb5e041e3d None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:03:53.811140 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbd1a94-0594-4bb1-85b7-c8eb5e041e3d None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:03:53.862010 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3959fa62-ab7d-4de4-a3e2-db15d39ce113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:03:53.862743 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3959fa62-ab7d-4de4-a3e2-db15d39ce113 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:03:53.862968 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3959fa62-ab7d-4de4-a3e2-db15d39ce113 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:03:53.863251 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3959fa62-ab7d-4de4-a3e2-db15d39ce113 None admin] PATH_INFO: `/v3/domains` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:03:53.864021 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3959fa62-ab7d-4de4-a3e2-db15d39ce113 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:03:53.864382 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3959fa62-ab7d-4de4-a3e2-db15d39ce113 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:03:53.866453 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3959fa62-ab7d-4de4-a3e2-db15d39ce113 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:03:53.888339 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 655/1279] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 6 20:03:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:03:53.894405 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca2a82b-1da8-4adf-aebc-3f6ce74810c2 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:03:53.937336 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:03:54.026386 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca2a82b-1da8-4adf-aebc-3f6ce74810c2 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:03:54.069054 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b81729-c29a-4a28-b73b-3ba5726055a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:03:54.070236 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b81729-c29a-4a28-b73b-3ba5726055a9 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:03:54.070773 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b81729-c29a-4a28-b73b-3ba5726055a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:03:54.071426 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b81729-c29a-4a28-b73b-3ba5726055a9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:03:54.090089 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b81729-c29a-4a28-b73b-3ba5726055a9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:03:54.093444 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b81729-c29a-4a28-b73b-3ba5726055a9 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:03:54.120857 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-60b81729-c29a-4a28-b73b-3ba5726055a9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 06 20:03:54.120857 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:03:54.120857 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:03:54.120857 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:03:54.120857 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:03:54.120857 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:03:54.120857 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:03:54.120857 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:03:54.120857 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:03:54.120857 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:03:54.120857 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:03:54.120857 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:03:54.120857 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 20:03:54.120857 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 20:03:54.120857 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 20:03:54.120857 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 20:03:54.122259 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:03:54.122259 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:03:54.122259 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 20:03:54.122259 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 20:03:54.122259 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 20:03:54.122259 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 20:03:54.122259 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:03:54.122259 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:03:54.122259 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:03:54.122259 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:03:54.122259 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:03:54.122259 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 20:03:54.122259 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:03:54.122259 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:03:54.122259 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:03:54.122259 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:03:54.122259 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:03:54.122259 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:03:54.123832 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:03:54.123832 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:03:54.123832 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 20:03:54.123832 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 20:03:54.123832 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 20:03:54.123832 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 20:03:54.123832 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 20:03:54.123832 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 20:03:54.123832 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 06 20:03:54.123832 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:03:54.123832 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 625/1280] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 20:03:53 2022] GET /identity/v3/users/placement => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 20:03:54.129878 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3959fa62-ab7d-4de4-a3e2-db15d39ce113 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:03:54.134756 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:03:54.266603 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3959fa62-ab7d-4de4-a3e2-db15d39ce113 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:03:54.308080 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343d8e98-2d86-4742-915e-75ce1886a4c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:03:54.308711 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343d8e98-2d86-4742-915e-75ce1886a4c1 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:03:54.309093 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343d8e98-2d86-4742-915e-75ce1886a4c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:03:54.309267 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343d8e98-2d86-4742-915e-75ce1886a4c1 None admin] PATH_INFO: `/v3/users` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:03:54.310057 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343d8e98-2d86-4742-915e-75ce1886a4c1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:03:54.310516 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343d8e98-2d86-4742-915e-75ce1886a4c1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:03:54.311994 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343d8e98-2d86-4742-915e-75ce1886a4c1 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:03:54.335235 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 656/1281] 10.222.0.20 () {56 vars in 1143 bytes} [Tue Sep 6 20:03:54 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:03:54.341680 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b81729-c29a-4a28-b73b-3ba5726055a9 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:03:54.453924 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:03:54.466469 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b81729-c29a-4a28-b73b-3ba5726055a9 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:03:54.508571 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac74ae2d-d1d0-462d-8919-5e00ce2bf833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:03:54.509365 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac74ae2d-d1d0-462d-8919-5e00ce2bf833 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:03:54.509792 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac74ae2d-d1d0-462d-8919-5e00ce2bf833 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:03:54.510166 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac74ae2d-d1d0-462d-8919-5e00ce2bf833 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:03:54.595010 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac74ae2d-d1d0-462d-8919-5e00ce2bf833 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:03:54.596523 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac74ae2d-d1d0-462d-8919-5e00ce2bf833 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:03:54.708999 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:03:54.722122 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 626/1282] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 6 20:03:54 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 20:03:54.729038 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343d8e98-2d86-4742-915e-75ce1886a4c1 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:03:54.802049 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:03:54.851604 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343d8e98-2d86-4742-915e-75ce1886a4c1 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:03:54.894025 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc16604-380f-4cf7-9f6c-d8891a8d05a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:03:54.894783 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc16604-380f-4cf7-9f6c-d8891a8d05a8 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:03:54.895068 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc16604-380f-4cf7-9f6c-d8891a8d05a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:03:54.895348 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc16604-380f-4cf7-9f6c-d8891a8d05a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:03:54.908977 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc16604-380f-4cf7-9f6c-d8891a8d05a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:03:54.910518 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc16604-380f-4cf7-9f6c-d8891a8d05a8 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:03:54.928251 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-6fc16604-380f-4cf7-9f6c-d8891a8d05a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 20:03:54.928251 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:03:54.928251 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:03:54.928251 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:03:54.928251 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:03:54.928251 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:03:54.928251 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:03:54.928251 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:03:54.928251 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:03:54.928251 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:03:54.928251 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:03:54.928251 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:03:54.928251 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 20:03:54.928251 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 20:03:54.928251 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 20:03:54.928251 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 20:03:54.929814 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:03:54.929814 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:03:54.929814 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:03:54.929814 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:03:54.929814 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:03:54.929814 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:03:54.929814 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:03:54.929814 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 20:03:54.929814 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:03:54.929814 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:03:54.929814 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:03:54.929814 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:03:54.929814 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:03:54.929814 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:03:54.929814 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:03:54.929814 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:03:54.929814 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 20:03:54.932314 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 20:03:54.932314 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 20:03:54.932314 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 20:03:54.933845 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 657/1283] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 20:03:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:03:54.936709 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac74ae2d-d1d0-462d-8919-5e00ce2bf833 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:03:55.067455 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac74ae2d-d1d0-462d-8919-5e00ce2bf833 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:03:55.113289 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cbb043-795e-425e-9fea-d6bd6cf69d70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:03:55.113837 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cbb043-795e-425e-9fea-d6bd6cf69d70 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:03:55.114115 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cbb043-795e-425e-9fea-d6bd6cf69d70 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:03:55.114385 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cbb043-795e-425e-9fea-d6bd6cf69d70 None admin] PATH_INFO: `/v3/domains` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:03:55.115091 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cbb043-795e-425e-9fea-d6bd6cf69d70 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:03:55.115386 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cbb043-795e-425e-9fea-d6bd6cf69d70 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:03:55.116880 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cbb043-795e-425e-9fea-d6bd6cf69d70 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:03:55.131857 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 627/1284] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 6 20:03:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:03:55.142116 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc16604-380f-4cf7-9f6c-d8891a8d05a8 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:03:55.288547 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc16604-380f-4cf7-9f6c-d8891a8d05a8 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:03:55.305488 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:03:55.329854 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddfb215-f9ec-49a5-b72d-e06c0a6eaf48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:03:55.330504 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddfb215-f9ec-49a5-b72d-e06c0a6eaf48 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:03:55.330775 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddfb215-f9ec-49a5-b72d-e06c0a6eaf48 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:03:55.331051 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddfb215-f9ec-49a5-b72d-e06c0a6eaf48 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:03:55.338414 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddfb215-f9ec-49a5-b72d-e06c0a6eaf48 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:03:55.339969 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddfb215-f9ec-49a5-b72d-e06c0a6eaf48 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:03:55.352617 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application [None req-fddfb215-f9ec-49a5-b72d-e06c0a6eaf48 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 20:03:55.352617 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:03:55.352617 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:03:55.352617 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:03:55.352617 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:03:55.352617 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:03:55.352617 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:03:55.352617 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:03:55.352617 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:03:55.352617 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:03:55.352617 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:03:55.352617 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:03:55.352617 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 20:03:55.352617 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 20:03:55.352617 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 20:03:55.354463 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 20:03:55.354463 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:03:55.354463 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:03:55.354463 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:03:55.354463 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:03:55.354463 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:03:55.354463 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:03:55.354463 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:03:55.354463 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application with Lock( Sep 06 20:03:55.354463 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:03:55.354463 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:03:55.354463 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:03:55.354463 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:03:55.354463 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:03:55.354463 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:03:55.354463 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:03:55.354463 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:03:55.355884 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 20:03:55.355884 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 20:03:55.355884 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 20:03:55.355884 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 20:03:55.355884 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 20:03:55.355884 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 20:03:55.355884 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 20:03:55.355884 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: ERROR keystone.server.flask.application Sep 06 20:03:55.355884 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 658/1285] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 6 20:03:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:03:55.360666 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cbb043-795e-425e-9fea-d6bd6cf69d70 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:03:55.420865 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:03:55.477634 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cbb043-795e-425e-9fea-d6bd6cf69d70 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:03:55.517441 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5d9d1a-b90d-424a-a0ea-abf6ab397203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:03:55.518216 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5d9d1a-b90d-424a-a0ea-abf6ab397203 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:03:55.518443 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5d9d1a-b90d-424a-a0ea-abf6ab397203 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:03:55.518719 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5d9d1a-b90d-424a-a0ea-abf6ab397203 None admin] PATH_INFO: `/v3/projects` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:03:55.519545 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5d9d1a-b90d-424a-a0ea-abf6ab397203 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:03:55.519775 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5d9d1a-b90d-424a-a0ea-abf6ab397203 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:03:55.521264 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5d9d1a-b90d-424a-a0ea-abf6ab397203 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:03:55.535898 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 628/1286] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Sep 6 20:03:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:03:55.543679 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddfb215-f9ec-49a5-b72d-e06c0a6eaf48 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:03:55.680553 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddfb215-f9ec-49a5-b72d-e06c0a6eaf48 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:03:55.724611 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29d84c0-7648-4c3c-ad2f-81e1e0c51aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:03:55.725258 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29d84c0-7648-4c3c-ad2f-81e1e0c51aa8 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:03:55.725572 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29d84c0-7648-4c3c-ad2f-81e1e0c51aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:03:55.725923 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29d84c0-7648-4c3c-ad2f-81e1e0c51aa8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:03:55.726779 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29d84c0-7648-4c3c-ad2f-81e1e0c51aa8 None admin] RBAC: Adding query filter params (role.id=e0ab41d8ae22474688b5361bcc75e0f0, scope.project.id=6aef43493bb1420a9e122313d8db7d5a, user.id=f57872de7bb74ea48d30e93e51830aba) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:03:55.727090 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29d84c0-7648-4c3c-ad2f-81e1e0c51aa8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:03:55.728774 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29d84c0-7648-4c3c-ad2f-81e1e0c51aa8 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:03:55.736696 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 659/1287] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Sep 6 20:03:55 2022] GET /identity/v3/role_assignments?user.id=f57872de7bb74ea48d30e93e51830aba&scope.project.id=6aef43493bb1420a9e122313d8db7d5a&role.id=e0ab41d8ae22474688b5361bcc75e0f0 => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 20:03:58.152792 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a55ca79-d625-449b-b62f-b364e3828434 None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:03:58.153495 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a55ca79-d625-449b-b62f-b364e3828434 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:03:58.153495 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a55ca79-d625-449b-b62f-b364e3828434 None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:03:58.155192 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 629/1288] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:03:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:03:58.161593 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d684a57-86c7-4107-91aa-12521a1b1089 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:03:58.162221 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d684a57-86c7-4107-91aa-12521a1b1089 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:03:58.162221 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d684a57-86c7-4107-91aa-12521a1b1089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:03:58.181463 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-9d684a57-86c7-4107-91aa-12521a1b1089 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:03:58.182818 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-9d684a57-86c7-4107-91aa-12521a1b1089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:03:58.220034 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:03:58.289302 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 660/1289] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:03:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 20:03:58.558952 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a55ca79-d625-449b-b62f-b364e3828434 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:03:58.678968 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a55ca79-d625-449b-b62f-b364e3828434 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:03:58.692026 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:03:58.721598 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64797529-4597-4ffe-b5ee-09b47d63e8c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:03:58.722296 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64797529-4597-4ffe-b5ee-09b47d63e8c3 None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:03:58.722582 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64797529-4597-4ffe-b5ee-09b47d63e8c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:03:58.722861 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64797529-4597-4ffe-b5ee-09b47d63e8c3 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:03:58.723482 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64797529-4597-4ffe-b5ee-09b47d63e8c3 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:03:58.724926 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64797529-4597-4ffe-b5ee-09b47d63e8c3 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:03:58.735516 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-64797529-4597-4ffe-b5ee-09b47d63e8c3 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 06 20:03:58.735516 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:03:58.735516 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 20:03:58.735516 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 20:03:58.735516 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 20:03:58.735516 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 20:03:58.735516 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 20:03:58.735516 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 20:03:58.735516 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 06 20:03:58.735516 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:03:58.735516 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 20:03:58.735516 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:03:58.735516 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:03:58.735516 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:03:58.735516 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:03:58.735516 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:03:58.735516 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:03:58.737785 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:03:58.737785 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:03:58.737785 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:03:58.737785 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:03:58.737785 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:03:58.737785 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:03:58.737785 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 20:03:58.737785 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 20:03:58.737785 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 20:03:58.737785 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 20:03:58.737785 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:03:58.737785 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:03:58.737785 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:03:58.737785 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:03:58.737785 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:03:58.737785 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:03:58.739564 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:03:58.739564 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 20:03:58.739564 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:03:58.739564 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:03:58.739564 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:03:58.739564 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:03:58.739564 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:03:58.739564 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:03:58.739564 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:03:58.739564 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:03:58.739564 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 20:03:58.739564 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 20:03:58.739564 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 06 20:03:58.739564 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:03:58.739564 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 630/1290] 10.222.0.20 () {56 vars in 1117 bytes} [Tue Sep 6 20:03:58 2022] GET /identity/v3/services/placement => generated 90 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:03:58.747791 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d684a57-86c7-4107-91aa-12521a1b1089 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:03:58.921016 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d684a57-86c7-4107-91aa-12521a1b1089 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:03:59.000520 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873dbb98-3aaf-4a88-bffb-8682f1bfadbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:03:59.001548 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873dbb98-3aaf-4a88-bffb-8682f1bfadbc None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:03:59.002068 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873dbb98-3aaf-4a88-bffb-8682f1bfadbc None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:03:59.002553 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873dbb98-3aaf-4a88-bffb-8682f1bfadbc None admin] PATH_INFO: `/v3/services` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:03:59.003738 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873dbb98-3aaf-4a88-bffb-8682f1bfadbc None admin] RBAC: Adding query filter params (name=placement) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:03:59.004287 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873dbb98-3aaf-4a88-bffb-8682f1bfadbc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:03:59.006992 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873dbb98-3aaf-4a88-bffb-8682f1bfadbc None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:03:59.013982 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 661/1291] 10.222.0.20 () {56 vars in 1116 bytes} [Tue Sep 6 20:03:58 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:03:59.020275 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64797529-4597-4ffe-b5ee-09b47d63e8c3 None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:03:59.141260 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64797529-4597-4ffe-b5ee-09b47d63e8c3 None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:03:59.185995 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da056be8-9046-48da-9536-efc74801233e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:03:59.187029 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da056be8-9046-48da-9536-efc74801233e None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:03:59.187029 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da056be8-9046-48da-9536-efc74801233e None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:03:59.187029 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da056be8-9046-48da-9536-efc74801233e None admin] PATH_INFO: `/v3/services` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:03:59.187697 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da056be8-9046-48da-9536-efc74801233e None admin] RBAC: Adding query filter params (type=placement) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:03:59.188007 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da056be8-9046-48da-9536-efc74801233e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:03:59.189414 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da056be8-9046-48da-9536-efc74801233e None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:03:59.196242 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 631/1292] 10.222.0.20 () {56 vars in 1116 bytes} [Tue Sep 6 20:03:59 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:04:01.463269 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b00c0f-2f5a-4ae6-bc32-5f71eacb99e9 None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:04:01.463877 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b00c0f-2f5a-4ae6-bc32-5f71eacb99e9 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:04:01.463877 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b00c0f-2f5a-4ae6-bc32-5f71eacb99e9 None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:04:01.465181 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 662/1293] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:04:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 20:04:01.471239 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfab79e3-bbe6-42eb-a0d8-6af83ed012fc None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:04:01.471808 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfab79e3-bbe6-42eb-a0d8-6af83ed012fc None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:04:01.471808 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfab79e3-bbe6-42eb-a0d8-6af83ed012fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:04:01.492855 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-cfab79e3-bbe6-42eb-a0d8-6af83ed012fc None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:04:01.494279 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-cfab79e3-bbe6-42eb-a0d8-6af83ed012fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:04:01.529879 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:04:01.573655 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 632/1294] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:04:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 20:04:01.844399 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b00c0f-2f5a-4ae6-bc32-5f71eacb99e9 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:04:01.885168 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:04:01.983558 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b00c0f-2f5a-4ae6-bc32-5f71eacb99e9 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:04:02.030730 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6796bca1-bad3-431a-a919-90761a1d0ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:04:02.031627 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6796bca1-bad3-431a-a919-90761a1d0ed2 None admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:04:02.031975 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6796bca1-bad3-431a-a919-90761a1d0ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:04:02.032585 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6796bca1-bad3-431a-a919-90761a1d0ed2 None admin] PATH_INFO: `/v3/services` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:04:02.033407 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6796bca1-bad3-431a-a919-90761a1d0ed2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:04:02.035065 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6796bca1-bad3-431a-a919-90761a1d0ed2 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:04:02.052208 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 663/1295] 10.222.0.20 () {60 vars in 1141 bytes} [Tue Sep 6 20:04:01 2022] POST /identity/v3/services => generated 246 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 20:04:04.366423 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915477c8-283d-4f67-a367-5b40d3be343d None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:04:04.366979 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915477c8-283d-4f67-a367-5b40d3be343d None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:04:04.366979 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915477c8-283d-4f67-a367-5b40d3be343d None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:04:04.369075 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 633/1296] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:04:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 20:04:04.375151 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e231795-9a0d-443f-b82a-f29bf15bc742 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:04:04.375721 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e231795-9a0d-443f-b82a-f29bf15bc742 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:04:04.375721 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e231795-9a0d-443f-b82a-f29bf15bc742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:04:04.395340 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-0e231795-9a0d-443f-b82a-f29bf15bc742 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:04:04.396618 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-0e231795-9a0d-443f-b82a-f29bf15bc742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:04:04.482075 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 664/1297] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:04:04 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 20:04:04.770414 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915477c8-283d-4f67-a367-5b40d3be343d None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:04:04.904028 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915477c8-283d-4f67-a367-5b40d3be343d None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:04:04.947324 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e212bfdd-8b72-44b6-b434-3f9aefaaea8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:04:04.947975 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e212bfdd-8b72-44b6-b434-3f9aefaaea8b None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:04:04.948257 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e212bfdd-8b72-44b6-b434-3f9aefaaea8b None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:04:04.948530 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e212bfdd-8b72-44b6-b434-3f9aefaaea8b None admin] PATH_INFO: `/v3/services/placement` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:04:04.949135 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e212bfdd-8b72-44b6-b434-3f9aefaaea8b None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:04:04.950623 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e212bfdd-8b72-44b6-b434-3f9aefaaea8b None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:04:04.960599 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-e212bfdd-8b72-44b6-b434-3f9aefaaea8b None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 06 20:04:04.960599 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:04:04.960599 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 20:04:04.960599 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 20:04:04.960599 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 20:04:04.960599 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 20:04:04.960599 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 20:04:04.960599 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 20:04:04.960599 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 06 20:04:04.960599 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:04:04.960599 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 20:04:04.960599 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:04:04.960599 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:04:04.960599 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:04:04.960599 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:04:04.960599 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:04:04.960599 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:04:04.962557 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:04:04.962557 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:04:04.962557 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:04:04.962557 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:04:04.962557 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:04:04.962557 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:04:04.962557 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 20:04:04.962557 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 20:04:04.962557 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 20:04:04.962557 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 20:04:04.962557 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:04:04.962557 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:04:04.962557 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:04:04.962557 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:04:04.962557 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:04:04.962557 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:04:04.963636 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:04:04.963636 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 20:04:04.963636 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:04:04.963636 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:04:04.963636 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:04:04.963636 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:04:04.963636 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:04:04.963636 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:04:04.963636 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:04:04.963636 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:04:04.963636 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 20:04:04.963636 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 20:04:04.963636 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 06 20:04:04.963636 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:04:04.963636 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 634/1298] 10.222.0.20 () {56 vars in 1117 bytes} [Tue Sep 6 20:04:04 2022] GET /identity/v3/services/placement => generated 90 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:04:04.968253 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e231795-9a0d-443f-b82a-f29bf15bc742 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:04:04.991360 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:04:05.099679 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e231795-9a0d-443f-b82a-f29bf15bc742 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:04:05.154062 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a6637b-3c61-4fcf-bbc9-d6d3ef9b9351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:04:05.154704 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a6637b-3c61-4fcf-bbc9-d6d3ef9b9351 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:04:05.155037 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a6637b-3c61-4fcf-bbc9-d6d3ef9b9351 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:04:05.155350 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a6637b-3c61-4fcf-bbc9-d6d3ef9b9351 None admin] PATH_INFO: `/v3/services` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:04:05.156532 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a6637b-3c61-4fcf-bbc9-d6d3ef9b9351 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:04:05.157106 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a6637b-3c61-4fcf-bbc9-d6d3ef9b9351 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:04:05.158626 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a6637b-3c61-4fcf-bbc9-d6d3ef9b9351 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:04:05.165288 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 665/1299] 10.222.0.20 () {56 vars in 1116 bytes} [Tue Sep 6 20:04:04 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 20:04:05.174616 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e212bfdd-8b72-44b6-b434-3f9aefaaea8b None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:04:05.214044 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:04:05.311048 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e212bfdd-8b72-44b6-b434-3f9aefaaea8b None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:04:05.357505 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114f224c-e193-4f93-bfad-58eb00ba359b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:04:05.358075 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114f224c-e193-4f93-bfad-58eb00ba359b None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:04:05.358359 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114f224c-e193-4f93-bfad-58eb00ba359b None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:04:05.358673 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114f224c-e193-4f93-bfad-58eb00ba359b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:04:05.359577 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114f224c-e193-4f93-bfad-58eb00ba359b None admin] RBAC: Adding query filter params (interface=public, service_id=2e73367dc9a54b2a9edbeaa834a43a53, region_id=RegionOne) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:04:05.359917 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114f224c-e193-4f93-bfad-58eb00ba359b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:04:05.361491 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114f224c-e193-4f93-bfad-58eb00ba359b None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:04:05.368404 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 635/1300] 10.222.0.20 () {56 vars in 1251 bytes} [Tue Sep 6 20:04:05 2022] GET /identity/v3/endpoints?service_id=2e73367dc9a54b2a9edbeaa834a43a53&interface=public®ion_id=RegionOne => generated 194 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:04:05.375133 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a6637b-3c61-4fcf-bbc9-d6d3ef9b9351 None admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:04:05.483214 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:04:05.508136 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a6637b-3c61-4fcf-bbc9-d6d3ef9b9351 None admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:04:05.553558 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebeb66b-be9b-485a-969b-e7febbbaa7a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:04:05.554280 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebeb66b-be9b-485a-969b-e7febbbaa7a8 None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:04:05.554604 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebeb66b-be9b-485a-969b-e7febbbaa7a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:04:05.555593 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebeb66b-be9b-485a-969b-e7febbbaa7a8 None admin] PATH_INFO: `/v3/services` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:04:05.556245 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebeb66b-be9b-485a-969b-e7febbbaa7a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:04:05.557726 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebeb66b-be9b-485a-969b-e7febbbaa7a8 None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:04:05.566787 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 666/1301] 10.222.0.20 () {56 vars in 1087 bytes} [Tue Sep 6 20:04:05 2022] GET /identity/v3/services => generated 2191 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:04:07.843189 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc2d8a2-9358-4669-99d9-73effd9352de None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:04:07.843774 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc2d8a2-9358-4669-99d9-73effd9352de None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:04:07.843774 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc2d8a2-9358-4669-99d9-73effd9352de None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:04:07.845131 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 636/1302] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:04:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:04:07.851037 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b7f7cf-71f5-43be-9ebc-307d5e4c68a7 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:04:07.852161 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b7f7cf-71f5-43be-9ebc-307d5e4c68a7 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:04:07.852161 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b7f7cf-71f5-43be-9ebc-307d5e4c68a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:04:07.871337 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-c0b7f7cf-71f5-43be-9ebc-307d5e4c68a7 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:04:07.872576 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-c0b7f7cf-71f5-43be-9ebc-307d5e4c68a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:04:07.969301 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 667/1303] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:04:07 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 20:04:08.251718 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc2d8a2-9358-4669-99d9-73effd9352de None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:04:08.363549 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:04:08.374602 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc2d8a2-9358-4669-99d9-73effd9352de None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:04:08.416396 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4869ce-c2da-4631-94fd-d92fec5a4bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:04:08.417100 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4869ce-c2da-4631-94fd-d92fec5a4bbe None admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:04:08.417423 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4869ce-c2da-4631-94fd-d92fec5a4bbe None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:04:08.417796 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4869ce-c2da-4631-94fd-d92fec5a4bbe None admin] PATH_INFO: `/v3/services/placement` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:04:08.418451 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4869ce-c2da-4631-94fd-d92fec5a4bbe None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:04:08.419914 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4869ce-c2da-4631-94fd-d92fec5a4bbe None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:04:08.430088 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application [None req-8a4869ce-c2da-4631-94fd-d92fec5a4bbe None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 06 20:04:08.430088 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:04:08.430088 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 20:04:08.430088 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 20:04:08.430088 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 20:04:08.430088 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 20:04:08.430088 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 20:04:08.430088 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 20:04:08.430088 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 06 20:04:08.430088 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:04:08.430088 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 20:04:08.430088 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:04:08.430088 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 20:04:08.430088 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 20:04:08.430088 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 20:04:08.430088 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 20:04:08.430088 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 20:04:08.431909 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 20:04:08.431909 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 20:04:08.431909 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 20:04:08.431909 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 20:04:08.431909 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 20:04:08.431909 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 20:04:08.431909 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 20:04:08.431909 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 20:04:08.431909 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 20:04:08.431909 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 20:04:08.431909 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 20:04:08.431909 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 20:04:08.431909 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 20:04:08.431909 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 20:04:08.431909 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 20:04:08.431909 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 20:04:08.434040 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 20:04:08.434040 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application with Lock( Sep 06 20:04:08.434040 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 20:04:08.434040 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self._enter() Sep 06 20:04:08.434040 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 20:04:08.434040 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 20:04:08.434040 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 20:04:08.434040 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application return self.creator() Sep 06 20:04:08.434040 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 20:04:08.434040 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application created_value = creator( Sep 06 20:04:08.434040 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 20:04:08.434040 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 20:04:08.434040 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 06 20:04:08.434040 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: ERROR keystone.server.flask.application Sep 06 20:04:08.434040 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 637/1304] 10.222.0.20 () {56 vars in 1117 bytes} [Tue Sep 6 20:04:08 2022] GET /identity/v3/services/placement => generated 90 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 20:04:08.438701 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b7f7cf-71f5-43be-9ebc-307d5e4c68a7 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:04:08.534953 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:04:08.563723 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b7f7cf-71f5-43be-9ebc-307d5e4c68a7 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:04:08.604771 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491b94e7-13a9-4068-b9c4-caeb65d7111a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:04:08.605501 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491b94e7-13a9-4068-b9c4-caeb65d7111a None admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:04:08.605711 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491b94e7-13a9-4068-b9c4-caeb65d7111a None admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:04:08.605989 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491b94e7-13a9-4068-b9c4-caeb65d7111a None admin] PATH_INFO: `/v3/services` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:04:08.606645 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491b94e7-13a9-4068-b9c4-caeb65d7111a None admin] RBAC: Adding query filter params (name=placement) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:04:08.606950 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491b94e7-13a9-4068-b9c4-caeb65d7111a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:04:08.608388 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491b94e7-13a9-4068-b9c4-caeb65d7111a None admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:04:08.614644 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 668/1305] 10.222.0.20 () {56 vars in 1116 bytes} [Tue Sep 6 20:04:08 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:04:08.621209 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4869ce-c2da-4631-94fd-d92fec5a4bbe None admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:04:08.771392 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4869ce-c2da-4631-94fd-d92fec5a4bbe None admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:04:08.843906 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab16fccf-795e-4ec4-9623-0df35624b937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:04:08.844735 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab16fccf-795e-4ec4-9623-0df35624b937 None admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:04:08.844910 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab16fccf-795e-4ec4-9623-0df35624b937 None admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:04:08.845232 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab16fccf-795e-4ec4-9623-0df35624b937 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:04:08.846018 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab16fccf-795e-4ec4-9623-0df35624b937 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:04:08.847500 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab16fccf-795e-4ec4-9623-0df35624b937 None admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:04:08.873247 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 638/1306] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:04:08 2022] POST /identity/v3/endpoints => generated 331 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 20:04:18.611132 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=30 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:04:18.866648 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:05:29.882461 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2888c23e-089b-40e9-b373-de262d465bff None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:05:29.883564 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2888c23e-089b-40e9-b373-de262d465bff None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:05:29.884191 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2888c23e-089b-40e9-b373-de262d465bff None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:05:29.887042 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 669/1307] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:05:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:05:29.894138 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f696576a-5957-4dec-ad2a-ec92276e9b08 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:05:29.894987 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f696576a-5957-4dec-ad2a-ec92276e9b08 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:05:29.895148 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f696576a-5957-4dec-ad2a-ec92276e9b08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:05:30.179119 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-f696576a-5957-4dec-ad2a-ec92276e9b08 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:05:30.181749 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-f696576a-5957-4dec-ad2a-ec92276e9b08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:05:30.336534 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 639/1308] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:05:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 444 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:05:30.866488 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07944cb6-662f-4ab2-9e4d-59d9398a4c67 None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:05:30.866971 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07944cb6-662f-4ab2-9e4d-59d9398a4c67 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:05:30.867471 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07944cb6-662f-4ab2-9e4d-59d9398a4c67 None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:05:30.870428 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 670/1309] 10.222.0.20 () {54 vars in 965 bytes} [Tue Sep 6 20:05:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:05:30.878334 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f8aae2-d25c-4453-a491-3b91a954fb94 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:05:30.878992 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f8aae2-d25c-4453-a491-3b91a954fb94 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:05:30.879526 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f8aae2-d25c-4453-a491-3b91a954fb94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:05:31.043890 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-16f8aae2-d25c-4453-a491-3b91a954fb94 None None] MFA Rules not processed for user `32882d38295046fea004a659a71bde58`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:05:31.047738 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-16f8aae2-d25c-4453-a491-3b91a954fb94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:05:31.095035 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:05:31.236285 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 640/1310] 10.222.0.20 () {58 vars in 1062 bytes} [Tue Sep 6 20:05:30 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 359 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:05:31.238046 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07944cb6-662f-4ab2-9e4d-59d9398a4c67 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:05:31.242537 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-07944cb6-662f-4ab2-9e4d-59d9398a4c67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:05:31.480783 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07944cb6-662f-4ab2-9e4d-59d9398a4c67 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:05:31.647646 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9db932-5b45-431f-8ce9-706d9984eb58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:05:31.648709 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9db932-5b45-431f-8ce9-706d9984eb58 service neutron] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:05:31.649484 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9db932-5b45-431f-8ce9-706d9984eb58 service neutron] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:05:31.650031 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9db932-5b45-431f-8ce9-706d9984eb58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:05:31.655342 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-dc9db932-5b45-431f-8ce9-706d9984eb58 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:05:31.779100 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:05:31.904337 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9db932-5b45-431f-8ce9-706d9984eb58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:05:31.906265 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9db932-5b45-431f-8ce9-706d9984eb58 service neutron] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:05:31.980790 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 671/1311] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:05:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 744 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:05:34.902458 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0525d5e5-7c77-4307-ae6f-d3486fb93d0f None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:05:34.904224 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0525d5e5-7c77-4307-ae6f-d3486fb93d0f None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:05:34.904224 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0525d5e5-7c77-4307-ae6f-d3486fb93d0f None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:05:34.905690 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 641/1312] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:05:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 20:05:34.912976 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df273ea-fb3b-4c2d-a975-fd33c10eaca6 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:05:34.913731 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df273ea-fb3b-4c2d-a975-fd33c10eaca6 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:05:34.913731 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df273ea-fb3b-4c2d-a975-fd33c10eaca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:05:35.020784 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-0df273ea-fb3b-4c2d-a975-fd33c10eaca6 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:05:35.021985 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-0df273ea-fb3b-4c2d-a975-fd33c10eaca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:05:35.147081 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 672/1313] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:05:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:05:35.484327 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ac180a-6a34-4b3d-9158-5cee8a7015dd None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:05:35.484610 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ac180a-6a34-4b3d-9158-5cee8a7015dd None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:05:35.484890 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ac180a-6a34-4b3d-9158-5cee8a7015dd None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:05:35.486345 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 642/1314] 10.222.0.20 () {54 vars in 965 bytes} [Tue Sep 6 20:05:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:05:35.493463 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a69c4d-f7ab-4da2-b513-d86d9a57cde8 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:05:35.493960 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a69c4d-f7ab-4da2-b513-d86d9a57cde8 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:05:35.494337 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a69c4d-f7ab-4da2-b513-d86d9a57cde8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:05:35.508264 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:05:35.593647 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-f2a69c4d-f7ab-4da2-b513-d86d9a57cde8 None None] MFA Rules not processed for user `32882d38295046fea004a659a71bde58`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:05:35.594932 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-f2a69c4d-f7ab-4da2-b513-d86d9a57cde8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:05:35.691181 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 673/1315] 10.222.0.20 () {58 vars in 1062 bytes} [Tue Sep 6 20:05:35 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:05:35.703455 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ac180a-6a34-4b3d-9158-5cee8a7015dd None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:05:35.705047 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-46ac180a-6a34-4b3d-9158-5cee8a7015dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:05:35.890834 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ac180a-6a34-4b3d-9158-5cee8a7015dd None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:05:36.035305 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:05:36.038624 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e07dd3a-0f56-41b1-9597-4213c30683e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:05:36.039226 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e07dd3a-0f56-41b1-9597-4213c30683e8 service neutron] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:05:36.039761 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e07dd3a-0f56-41b1-9597-4213c30683e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:05:36.040100 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e07dd3a-0f56-41b1-9597-4213c30683e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:05:36.042356 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-2e07dd3a-0f56-41b1-9597-4213c30683e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:05:36.193081 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e07dd3a-0f56-41b1-9597-4213c30683e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:05:36.195560 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e07dd3a-0f56-41b1-9597-4213c30683e8 service neutron] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:05:36.232116 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 643/1316] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:05:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 530 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:05:39.242593 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd20d54-c6a0-4caf-b6a7-b4ac5fe0d3bc None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:05:39.243228 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd20d54-c6a0-4caf-b6a7-b4ac5fe0d3bc None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:05:39.243228 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd20d54-c6a0-4caf-b6a7-b4ac5fe0d3bc None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:05:39.244554 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 674/1317] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:05:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:05:39.251220 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780e6f61-bbb8-47cd-8370-866389bde1d8 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:05:39.251996 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780e6f61-bbb8-47cd-8370-866389bde1d8 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:05:39.252154 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780e6f61-bbb8-47cd-8370-866389bde1d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:05:39.334009 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-780e6f61-bbb8-47cd-8370-866389bde1d8 None None] MFA Rules not processed for user `a729c9f9f70c41eab4ccf79f7ed5ff84`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:05:39.335422 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-780e6f61-bbb8-47cd-8370-866389bde1d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:05:39.405420 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:05:39.460824 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 644/1318] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:05:39 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:05:39.768425 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd20d54-c6a0-4caf-b6a7-b4ac5fe0d3bc None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:05:39.771312 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-fcd20d54-c6a0-4caf-b6a7-b4ac5fe0d3bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:05:39.929973 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd20d54-c6a0-4caf-b6a7-b4ac5fe0d3bc None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:05:39.936162 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:05:40.061778 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c1e700-6081-4bf8-9495-fe7e8504bdbe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:05:40.062438 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c1e700-6081-4bf8-9495-fe7e8504bdbe service neutron] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:05:40.062719 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c1e700-6081-4bf8-9495-fe7e8504bdbe service neutron] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:05:40.063065 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c1e700-6081-4bf8-9495-fe7e8504bdbe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:05:40.064952 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-99c1e700-6081-4bf8-9495-fe7e8504bdbe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:05:40.197697 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c1e700-6081-4bf8-9495-fe7e8504bdbe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:05:40.199324 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c1e700-6081-4bf8-9495-fe7e8504bdbe service neutron] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:05:40.220638 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 675/1319] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:05:39 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:05:43.365246 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ff6be4-b554-4e0f-8fc2-5c089fc62a96 None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:05:43.365944 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ff6be4-b554-4e0f-8fc2-5c089fc62a96 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:05:43.365944 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ff6be4-b554-4e0f-8fc2-5c089fc62a96 None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:05:43.367361 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 645/1320] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:05:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 20:05:43.374119 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5d2d0c-3365-4015-98b8-644353c6667c None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:05:43.374927 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5d2d0c-3365-4015-98b8-644353c6667c None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:05:43.375118 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5d2d0c-3365-4015-98b8-644353c6667c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:05:43.427091 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:05:43.460615 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-4a5d2d0c-3365-4015-98b8-644353c6667c None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:05:43.461955 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-4a5d2d0c-3365-4015-98b8-644353c6667c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:05:43.622060 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 676/1321] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:05:43 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:05:43.936967 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ff6be4-b554-4e0f-8fc2-5c089fc62a96 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:05:43.945683 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-12ff6be4-b554-4e0f-8fc2-5c089fc62a96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:05:44.107954 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ff6be4-b554-4e0f-8fc2-5c089fc62a96 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:05:44.211422 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5e5a00-0773-448e-abaf-f2202be811b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:05:44.211983 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5e5a00-0773-448e-abaf-f2202be811b4 service neutron] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:05:44.212244 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5e5a00-0773-448e-abaf-f2202be811b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:05:44.212508 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5e5a00-0773-448e-abaf-f2202be811b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:05:44.214259 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-db5e5a00-0773-448e-abaf-f2202be811b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:05:44.223123 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:05:44.351064 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5e5a00-0773-448e-abaf-f2202be811b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:05:44.352529 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5e5a00-0773-448e-abaf-f2202be811b4 service neutron] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:05:44.376003 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 646/1322] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:05:43 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:05:47.763187 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4466c9fb-f677-4f9b-8839-9647400a0de8 None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:05:47.764031 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4466c9fb-f677-4f9b-8839-9647400a0de8 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:05:47.764189 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4466c9fb-f677-4f9b-8839-9647400a0de8 None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:05:47.765843 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 677/1323] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:05:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:05:47.772799 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0e52da-7f84-4fda-bcd7-754e15782462 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:05:47.773399 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0e52da-7f84-4fda-bcd7-754e15782462 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:05:47.773399 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0e52da-7f84-4fda-bcd7-754e15782462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:05:47.860403 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-dd0e52da-7f84-4fda-bcd7-754e15782462 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:05:47.861753 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-dd0e52da-7f84-4fda-bcd7-754e15782462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:05:47.942729 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:05:47.998425 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 647/1324] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:05:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:05:48.288000 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4466c9fb-f677-4f9b-8839-9647400a0de8 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:05:48.289987 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-4466c9fb-f677-4f9b-8839-9647400a0de8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:05:48.418852 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4466c9fb-f677-4f9b-8839-9647400a0de8 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:05:48.433445 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:05:48.512276 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9636684-f551-4bc8-8ec7-26edb7287668 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:05:48.512906 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9636684-f551-4bc8-8ec7-26edb7287668 service neutron] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:05:48.513187 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9636684-f551-4bc8-8ec7-26edb7287668 service neutron] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:05:48.513460 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9636684-f551-4bc8-8ec7-26edb7287668 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:05:48.515191 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-f9636684-f551-4bc8-8ec7-26edb7287668 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:05:48.664836 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9636684-f551-4bc8-8ec7-26edb7287668 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:05:48.666531 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9636684-f551-4bc8-8ec7-26edb7287668 service neutron] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:05:48.692952 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 678/1325] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:05:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:05:51.314970 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4308a84c-2d75-4874-9561-6a96ef5ce92a None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:05:51.315563 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4308a84c-2d75-4874-9561-6a96ef5ce92a None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:05:51.315943 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4308a84c-2d75-4874-9561-6a96ef5ce92a None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:05:51.323571 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 648/1326] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:05:51 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:05:51.337642 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acac3e0c-24fd-4cad-8510-763affabbfe8 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:05:51.338480 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acac3e0c-24fd-4cad-8510-763affabbfe8 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:05:51.338642 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acac3e0c-24fd-4cad-8510-763affabbfe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:05:51.407747 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:05:51.431545 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-acac3e0c-24fd-4cad-8510-763affabbfe8 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:05:51.432831 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-acac3e0c-24fd-4cad-8510-763affabbfe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:05:51.583510 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 679/1327] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:05:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:05:51.901702 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4308a84c-2d75-4874-9561-6a96ef5ce92a None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:05:51.905056 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-4308a84c-2d75-4874-9561-6a96ef5ce92a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:05:52.121832 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4308a84c-2d75-4874-9561-6a96ef5ce92a None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:05:52.272427 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b1635c-e911-4f39-9f03-ed8ccbdd14fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:05:52.273191 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b1635c-e911-4f39-9f03-ed8ccbdd14fc service neutron] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:05:52.273579 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b1635c-e911-4f39-9f03-ed8ccbdd14fc service neutron] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:05:52.274244 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b1635c-e911-4f39-9f03-ed8ccbdd14fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:05:52.276248 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-a9b1635c-e911-4f39-9f03-ed8ccbdd14fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:05:52.299026 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:05:52.435706 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b1635c-e911-4f39-9f03-ed8ccbdd14fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:05:52.437671 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b1635c-e911-4f39-9f03-ed8ccbdd14fc service neutron] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:05:52.461797 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 649/1328] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:05:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 561 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:05:55.051016 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71657a62-464a-48ae-b0ef-b2c49725e714 None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:05:55.051776 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71657a62-464a-48ae-b0ef-b2c49725e714 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:05:55.051776 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71657a62-464a-48ae-b0ef-b2c49725e714 None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:05:55.053325 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 680/1329] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:05:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 20:05:55.061487 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac3ecff-976c-467e-abbd-43b0a1b678d3 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:05:55.062291 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac3ecff-976c-467e-abbd-43b0a1b678d3 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:05:55.062466 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac3ecff-976c-467e-abbd-43b0a1b678d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:05:55.183940 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-2ac3ecff-976c-467e-abbd-43b0a1b678d3 None None] MFA Rules not processed for user `a729c9f9f70c41eab4ccf79f7ed5ff84`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:05:55.186649 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-2ac3ecff-976c-467e-abbd-43b0a1b678d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:05:55.311239 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:05:55.350304 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 650/1330] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:05:55 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:05:55.715928 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71657a62-464a-48ae-b0ef-b2c49725e714 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:05:55.717494 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-71657a62-464a-48ae-b0ef-b2c49725e714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:05:55.866533 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71657a62-464a-48ae-b0ef-b2c49725e714 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:05:55.897911 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:05:56.007510 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae6b589-e539-4d75-9bd7-d73470a18fce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:05:56.008539 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae6b589-e539-4d75-9bd7-d73470a18fce service neutron] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:05:56.009144 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae6b589-e539-4d75-9bd7-d73470a18fce service neutron] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:05:56.009781 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae6b589-e539-4d75-9bd7-d73470a18fce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:05:56.013848 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-cae6b589-e539-4d75-9bd7-d73470a18fce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:05:56.195617 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae6b589-e539-4d75-9bd7-d73470a18fce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:05:56.198687 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae6b589-e539-4d75-9bd7-d73470a18fce service neutron] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:05:56.231809 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 681/1331] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:05:55 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 517 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 20:05:59.170197 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b6a401-ea95-43cb-8118-16f4c82b28f9 None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:05:59.170792 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b6a401-ea95-43cb-8118-16f4c82b28f9 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:05:59.170792 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b6a401-ea95-43cb-8118-16f4c82b28f9 None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:05:59.172176 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 651/1332] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:05:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:05:59.178108 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495a5a89-e8e6-4937-bff0-27ddd9b9bef1 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:05:59.178880 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495a5a89-e8e6-4937-bff0-27ddd9b9bef1 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:05:59.178880 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495a5a89-e8e6-4937-bff0-27ddd9b9bef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:05:59.253531 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:05:59.257156 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-495a5a89-e8e6-4937-bff0-27ddd9b9bef1 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:05:59.259314 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-495a5a89-e8e6-4937-bff0-27ddd9b9bef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:05:59.374903 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 682/1333] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:05:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:05:59.662526 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b6a401-ea95-43cb-8118-16f4c82b28f9 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:05:59.664156 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-c7b6a401-ea95-43cb-8118-16f4c82b28f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:05:59.782462 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b6a401-ea95-43cb-8118-16f4c82b28f9 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:05:59.873594 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279a6489-872a-49fe-9fa5-6852c48a29c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:05:59.874248 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279a6489-872a-49fe-9fa5-6852c48a29c8 service neutron] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:05:59.874528 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279a6489-872a-49fe-9fa5-6852c48a29c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:05:59.874803 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279a6489-872a-49fe-9fa5-6852c48a29c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:05:59.876694 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-279a6489-872a-49fe-9fa5-6852c48a29c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:05:59.909344 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:06:00.009392 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279a6489-872a-49fe-9fa5-6852c48a29c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:06:00.010889 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279a6489-872a-49fe-9fa5-6852c48a29c8 service neutron] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:06:00.032115 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 652/1334] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:05:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:06:02.422699 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67950b75-8001-4c30-be12-ae90ddf9b19d None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:06:02.423420 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67950b75-8001-4c30-be12-ae90ddf9b19d None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:06:02.423420 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67950b75-8001-4c30-be12-ae90ddf9b19d None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:06:02.424900 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 683/1335] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:06:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:06:02.431251 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c47a4e5-baa5-4d2b-8665-28e45b60e2ac None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:06:02.432111 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c47a4e5-baa5-4d2b-8665-28e45b60e2ac None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:06:02.432111 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c47a4e5-baa5-4d2b-8665-28e45b60e2ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:06:02.529994 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-3c47a4e5-baa5-4d2b-8665-28e45b60e2ac None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:06:02.531765 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-3c47a4e5-baa5-4d2b-8665-28e45b60e2ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:06:02.629904 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:06:02.660154 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 653/1336] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:06:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:06:02.961885 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67950b75-8001-4c30-be12-ae90ddf9b19d None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:06:02.964242 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-67950b75-8001-4c30-be12-ae90ddf9b19d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:06:03.136363 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67950b75-8001-4c30-be12-ae90ddf9b19d None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:06:03.180533 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:06:03.252065 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70e9aa5-7ea4-4804-96c1-d73822638cb7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:06:03.252693 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70e9aa5-7ea4-4804-96c1-d73822638cb7 service neutron] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:06:03.252976 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70e9aa5-7ea4-4804-96c1-d73822638cb7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:06:03.253252 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70e9aa5-7ea4-4804-96c1-d73822638cb7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:06:03.255037 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-d70e9aa5-7ea4-4804-96c1-d73822638cb7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:06:03.407120 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70e9aa5-7ea4-4804-96c1-d73822638cb7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:06:03.408532 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70e9aa5-7ea4-4804-96c1-d73822638cb7 service neutron] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:06:03.430250 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 684/1337] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:06:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:06:06.241712 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de7c0e5-841f-4f5e-8466-e24337017264 None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:06:06.242340 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de7c0e5-841f-4f5e-8466-e24337017264 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:06:06.242649 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de7c0e5-841f-4f5e-8466-e24337017264 None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:06:06.245220 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 654/1338] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:06:06 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:06:06.252047 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e00933c-d6ee-4aa1-a7cd-280698f757ee None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:06:06.252739 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e00933c-d6ee-4aa1-a7cd-280698f757ee None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:06:06.252739 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e00933c-d6ee-4aa1-a7cd-280698f757ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:06:06.336922 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-9e00933c-d6ee-4aa1-a7cd-280698f757ee None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:06:06.338276 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-9e00933c-d6ee-4aa1-a7cd-280698f757ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:06:06.348657 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:06:06.458728 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 685/1339] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:06:06 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:06:06.755857 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de7c0e5-841f-4f5e-8466-e24337017264 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:06:06.757496 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-9de7c0e5-841f-4f5e-8466-e24337017264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:06:06.887870 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de7c0e5-841f-4f5e-8466-e24337017264 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:06:07.012149 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282c0031-b576-4068-84ed-4047bd47fdec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:06:07.012913 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282c0031-b576-4068-84ed-4047bd47fdec service neutron] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:06:07.013285 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282c0031-b576-4068-84ed-4047bd47fdec service neutron] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:06:07.018480 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282c0031-b576-4068-84ed-4047bd47fdec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:06:07.020501 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-282c0031-b576-4068-84ed-4047bd47fdec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:06:07.073516 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:06:07.171709 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282c0031-b576-4068-84ed-4047bd47fdec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:06:07.173274 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282c0031-b576-4068-84ed-4047bd47fdec service neutron] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:06:07.195888 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 655/1340] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:06:06 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:06:13.036657 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79908083-fc8d-4c35-b54b-33559c99c744 None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:06:13.037227 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79908083-fc8d-4c35-b54b-33559c99c744 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:06:13.037227 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79908083-fc8d-4c35-b54b-33559c99c744 None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:06:13.039050 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 686/1341] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:06:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:06:13.045037 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae74d85-b7c7-426a-a15a-a3d3234eb694 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:06:13.045700 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae74d85-b7c7-426a-a15a-a3d3234eb694 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:06:13.045700 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae74d85-b7c7-426a-a15a-a3d3234eb694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:06:13.134121 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-1ae74d85-b7c7-426a-a15a-a3d3234eb694 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:06:13.135404 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-1ae74d85-b7c7-426a-a15a-a3d3234eb694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:06:13.290375 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:06:13.312921 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 656/1342] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:06:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:06:13.634317 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79908083-fc8d-4c35-b54b-33559c99c744 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:06:13.637368 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-79908083-fc8d-4c35-b54b-33559c99c744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:06:13.856496 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79908083-fc8d-4c35-b54b-33559c99c744 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:06:13.918849 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:06:13.987627 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ab3632-9250-4aec-87e8-b651360998a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:06:13.988634 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ab3632-9250-4aec-87e8-b651360998a3 service neutron] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:06:13.989093 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ab3632-9250-4aec-87e8-b651360998a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:06:13.989561 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ab3632-9250-4aec-87e8-b651360998a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:06:13.993148 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-a6ab3632-9250-4aec-87e8-b651360998a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:06:14.172445 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ab3632-9250-4aec-87e8-b651360998a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:06:14.173881 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ab3632-9250-4aec-87e8-b651360998a3 service neutron] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:06:14.198083 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 687/1343] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:06:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 565 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:06:18.102845 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9008641c-b599-4702-a37c-eaf958aff029 None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:06:18.103432 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9008641c-b599-4702-a37c-eaf958aff029 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:06:18.103901 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9008641c-b599-4702-a37c-eaf958aff029 None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:06:18.106815 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 657/1344] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:06:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:06:18.113095 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f50a989-6cbf-405d-92b4-71b2e0da4319 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:06:18.113754 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f50a989-6cbf-405d-92b4-71b2e0da4319 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:06:18.113754 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f50a989-6cbf-405d-92b4-71b2e0da4319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:06:18.213338 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-1f50a989-6cbf-405d-92b4-71b2e0da4319 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:06:18.215555 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-1f50a989-6cbf-405d-92b4-71b2e0da4319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:06:18.241583 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:06:18.354972 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 688/1345] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:06:18 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:06:18.672643 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9008641c-b599-4702-a37c-eaf958aff029 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:06:18.674631 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-9008641c-b599-4702-a37c-eaf958aff029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:06:18.882832 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9008641c-b599-4702-a37c-eaf958aff029 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:06:19.001453 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1abf51-746f-4dcb-b3d6-7fdd25294b83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:06:19.002345 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1abf51-746f-4dcb-b3d6-7fdd25294b83 service neutron] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:06:19.002729 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1abf51-746f-4dcb-b3d6-7fdd25294b83 service neutron] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:06:19.003102 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1abf51-746f-4dcb-b3d6-7fdd25294b83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:06:19.005028 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-dd1abf51-746f-4dcb-b3d6-7fdd25294b83 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:06:19.070046 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:06:19.191198 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1abf51-746f-4dcb-b3d6-7fdd25294b83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:06:19.193889 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1abf51-746f-4dcb-b3d6-7fdd25294b83 service neutron] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:06:19.232336 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 658/1346] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:06:18 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 560 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 20:06:24.629651 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a159ebe-4560-4d52-9931-244661e23e25 None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:06:24.630306 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a159ebe-4560-4d52-9931-244661e23e25 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:06:24.630306 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a159ebe-4560-4d52-9931-244661e23e25 None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:06:24.633011 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 689/1347] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:06:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:06:24.640687 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c0450b-4aea-4b35-b246-9c4c4d5d8bff None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:06:24.641261 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c0450b-4aea-4b35-b246-9c4c4d5d8bff None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:06:24.641522 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c0450b-4aea-4b35-b246-9c4c4d5d8bff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:06:24.751093 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-d6c0450b-4aea-4b35-b246-9c4c4d5d8bff None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:06:24.754034 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-d6c0450b-4aea-4b35-b246-9c4c4d5d8bff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:06:24.940767 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:06:24.947315 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 659/1348] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:06:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:06:25.365371 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a159ebe-4560-4d52-9931-244661e23e25 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:06:25.366980 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-1a159ebe-4560-4d52-9931-244661e23e25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:06:25.527245 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a159ebe-4560-4d52-9931-244661e23e25 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:06:25.635160 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:06:25.692645 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288284a1-7f37-4554-b29a-17801f8cdb08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:06:25.693424 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288284a1-7f37-4554-b29a-17801f8cdb08 service neutron] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:06:25.693867 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288284a1-7f37-4554-b29a-17801f8cdb08 service neutron] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:06:25.694245 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288284a1-7f37-4554-b29a-17801f8cdb08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:06:25.696217 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-288284a1-7f37-4554-b29a-17801f8cdb08 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:06:25.873780 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288284a1-7f37-4554-b29a-17801f8cdb08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:06:25.875354 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288284a1-7f37-4554-b29a-17801f8cdb08 service neutron] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:06:25.900710 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 690/1349] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:06:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 537 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:06:32.244867 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d919cf-51da-48f1-b432-a10303fbea03 None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:06:32.246526 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d919cf-51da-48f1-b432-a10303fbea03 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:06:32.247167 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d919cf-51da-48f1-b432-a10303fbea03 None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:06:32.250540 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 660/1350] 10.222.0.20 () {54 vars in 921 bytes} [Tue Sep 6 20:06:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:06:32.256571 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a97a6d7-e75e-4831-af59-430626a1e4f3 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:06:32.257192 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a97a6d7-e75e-4831-af59-430626a1e4f3 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:06:32.257192 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a97a6d7-e75e-4831-af59-430626a1e4f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:06:32.396054 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-2a97a6d7-e75e-4831-af59-430626a1e4f3 None None] MFA Rules not processed for user `f57872de7bb74ea48d30e93e51830aba`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:06:32.397323 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-2a97a6d7-e75e-4831-af59-430626a1e4f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:06:32.424194 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:06:32.603420 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 691/1351] 10.222.0.20 () {58 vars in 1018 bytes} [Tue Sep 6 20:06:32 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 348 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:06:38.453121 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc03fe0-d12c-4e8e-8672-1a9e735bf7f5 None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:06:38.454992 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc03fe0-d12c-4e8e-8672-1a9e735bf7f5 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:06:38.455474 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc03fe0-d12c-4e8e-8672-1a9e735bf7f5 None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:06:38.458418 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 661/1352] 10.222.0.20 () {54 vars in 921 bytes} [Tue Sep 6 20:06:38 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:06:38.470970 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8368df-efa2-4709-be99-2efbd54caf77 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:06:38.471813 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8368df-efa2-4709-be99-2efbd54caf77 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:06:38.471995 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8368df-efa2-4709-be99-2efbd54caf77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:06:38.607642 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-5b8368df-efa2-4709-be99-2efbd54caf77 None None] MFA Rules not processed for user `f57872de7bb74ea48d30e93e51830aba`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:06:38.609972 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-5b8368df-efa2-4709-be99-2efbd54caf77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:06:38.745737 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:06:38.800134 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 692/1353] 10.222.0.20 () {58 vars in 1018 bytes} [Tue Sep 6 20:06:38 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:06:40.038407 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d45f05a-674c-41b0-aacd-cf8207669327 None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:06:40.038923 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d45f05a-674c-41b0-aacd-cf8207669327 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:06:40.038923 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d45f05a-674c-41b0-aacd-cf8207669327 None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:06:40.040489 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 662/1354] 10.222.0.20 () {54 vars in 921 bytes} [Tue Sep 6 20:06:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:06:40.047478 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9ad28f-fcea-45a4-a07c-5ec6631ed3f3 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:06:40.048126 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9ad28f-fcea-45a4-a07c-5ec6631ed3f3 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:06:40.048126 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9ad28f-fcea-45a4-a07c-5ec6631ed3f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:06:40.170084 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-5a9ad28f-fcea-45a4-a07c-5ec6631ed3f3 None None] MFA Rules not processed for user `f57872de7bb74ea48d30e93e51830aba`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:06:40.171360 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-5a9ad28f-fcea-45a4-a07c-5ec6631ed3f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:06:40.327080 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 693/1355] 10.222.0.20 () {58 vars in 1018 bytes} [Tue Sep 6 20:06:40 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:06:44.199146 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93b5d38-4cef-4db1-b4b0-6accb526f689 None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:06:44.199748 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93b5d38-4cef-4db1-b4b0-6accb526f689 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:06:44.199748 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93b5d38-4cef-4db1-b4b0-6accb526f689 None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:06:44.201216 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 663/1356] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:06:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:06:44.208492 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61c1c08-a977-41ec-85bf-0b90747ace15 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:06:44.209673 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61c1c08-a977-41ec-85bf-0b90747ace15 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:06:44.210017 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61c1c08-a977-41ec-85bf-0b90747ace15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:06:44.285260 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:06:44.339788 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-b61c1c08-a977-41ec-85bf-0b90747ace15 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:06:44.340041 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-b61c1c08-a977-41ec-85bf-0b90747ace15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:06:44.515357 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 694/1357] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:06:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:06:44.843854 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f69d450-2adf-4b37-8ed2-41b8d4cd5514 None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:06:44.844568 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f69d450-2adf-4b37-8ed2-41b8d4cd5514 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:06:44.845136 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f69d450-2adf-4b37-8ed2-41b8d4cd5514 None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:06:44.846970 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 664/1358] 10.222.0.20 () {54 vars in 962 bytes} [Tue Sep 6 20:06:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:06:44.863479 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4fae03-160d-42a6-ae6d-2f2616c01dc9 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:06:44.864073 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4fae03-160d-42a6-ae6d-2f2616c01dc9 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:06:44.864564 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4fae03-160d-42a6-ae6d-2f2616c01dc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:06:44.988611 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-bc4fae03-160d-42a6-ae6d-2f2616c01dc9 None None] MFA Rules not processed for user `5776cbf4f6714b2083c3ebdd6d98cc87`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:06:44.990027 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-bc4fae03-160d-42a6-ae6d-2f2616c01dc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:06:45.065077 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:06:45.168875 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 695/1359] 10.222.0.20 () {58 vars in 1059 bytes} [Tue Sep 6 20:06:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:06:45.178129 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f69d450-2adf-4b37-8ed2-41b8d4cd5514 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:06:45.180059 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-1f69d450-2adf-4b37-8ed2-41b8d4cd5514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:06:45.378983 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f69d450-2adf-4b37-8ed2-41b8d4cd5514 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:06:45.504432 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:06:45.582538 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8728141f-34aa-48f0-9e63-9b3102dac717 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5776cbf4f6714b2083c3ebdd6d98cc87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:06:45.583971 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8728141f-34aa-48f0-9e63-9b3102dac717 service nova] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:06:45.584278 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8728141f-34aa-48f0-9e63-9b3102dac717 service nova] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:06:45.584704 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8728141f-34aa-48f0-9e63-9b3102dac717 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:06:45.586456 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-8728141f-34aa-48f0-9e63-9b3102dac717 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:06:45.728543 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8728141f-34aa-48f0-9e63-9b3102dac717 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:06:45.730206 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8728141f-34aa-48f0-9e63-9b3102dac717 service nova] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:06:45.759839 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 665/1360] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:06:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 582 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 20:06:48.677794 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e468fe24-7c15-4598-9390-a7e17935c8aa None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:06:48.678614 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e468fe24-7c15-4598-9390-a7e17935c8aa None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:06:48.678782 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e468fe24-7c15-4598-9390-a7e17935c8aa None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:06:48.681333 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 696/1361] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:06:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:06:48.691518 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f02b1f5-9781-4646-8fbc-86c2cc786089 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:06:48.692356 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f02b1f5-9781-4646-8fbc-86c2cc786089 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:06:48.692356 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f02b1f5-9781-4646-8fbc-86c2cc786089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:06:48.811546 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-4f02b1f5-9781-4646-8fbc-86c2cc786089 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:06:48.813012 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-4f02b1f5-9781-4646-8fbc-86c2cc786089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:06:48.872500 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:06:48.968917 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 666/1362] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:06:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:06:49.381353 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e468fe24-7c15-4598-9390-a7e17935c8aa None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:06:49.383408 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-e468fe24-7c15-4598-9390-a7e17935c8aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:06:49.662487 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e468fe24-7c15-4598-9390-a7e17935c8aa None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:06:49.677036 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:06:49.922347 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab9df1b-60cb-41f2-906d-71f69addf673 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5776cbf4f6714b2083c3ebdd6d98cc87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:06:49.923706 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab9df1b-60cb-41f2-906d-71f69addf673 service nova] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:06:49.924127 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab9df1b-60cb-41f2-906d-71f69addf673 service nova] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:06:49.924473 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab9df1b-60cb-41f2-906d-71f69addf673 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:06:49.928154 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-fab9df1b-60cb-41f2-906d-71f69addf673 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:06:50.099188 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab9df1b-60cb-41f2-906d-71f69addf673 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:06:50.101546 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab9df1b-60cb-41f2-906d-71f69addf673 service nova] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:06:50.128980 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 697/1363] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:06:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 748 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:06:52.895625 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87861560-aaae-4d47-aba3-56b96a9a2da9 None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:06:52.896465 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87861560-aaae-4d47-aba3-56b96a9a2da9 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:06:52.896465 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87861560-aaae-4d47-aba3-56b96a9a2da9 None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:06:52.898148 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 667/1364] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:06:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:06:52.908579 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9be2529-c8b8-46ee-9802-491b71add092 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:06:52.909414 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9be2529-c8b8-46ee-9802-491b71add092 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:06:52.909578 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9be2529-c8b8-46ee-9802-491b71add092 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:06:52.943457 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:06:53.092175 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-d9be2529-c8b8-46ee-9802-491b71add092 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:06:53.095494 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-d9be2529-c8b8-46ee-9802-491b71add092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:06:53.399365 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 698/1365] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:06:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 493 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:06:53.926287 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b6c9fc-cfea-408c-80d8-8b9413a8cbdf None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:06:53.927167 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b6c9fc-cfea-408c-80d8-8b9413a8cbdf None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:06:53.927167 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b6c9fc-cfea-408c-80d8-8b9413a8cbdf None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:06:53.928716 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 668/1366] 10.222.0.20 () {54 vars in 962 bytes} [Tue Sep 6 20:06:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 20:06:53.936170 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d2fdf3-58f6-4c1a-8ffd-ed9d532bc4a3 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:06:53.936943 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d2fdf3-58f6-4c1a-8ffd-ed9d532bc4a3 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:06:53.936943 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d2fdf3-58f6-4c1a-8ffd-ed9d532bc4a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:06:54.055985 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-b6d2fdf3-58f6-4c1a-8ffd-ed9d532bc4a3 None None] MFA Rules not processed for user `5776cbf4f6714b2083c3ebdd6d98cc87`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:06:54.057270 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-b6d2fdf3-58f6-4c1a-8ffd-ed9d532bc4a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:06:54.108187 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:06:54.240766 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 699/1367] 10.222.0.20 () {58 vars in 1059 bytes} [Tue Sep 6 20:06:53 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:06:54.245043 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b6c9fc-cfea-408c-80d8-8b9413a8cbdf None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:06:54.246780 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-d4b6c9fc-cfea-408c-80d8-8b9413a8cbdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:06:54.529983 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b6c9fc-cfea-408c-80d8-8b9413a8cbdf None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:06:54.573521 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:06:54.807220 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434f0e0d-de92-4747-8174-63c8cc247b11 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5776cbf4f6714b2083c3ebdd6d98cc87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:06:54.808594 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434f0e0d-de92-4747-8174-63c8cc247b11 service nova] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:06:54.809086 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434f0e0d-de92-4747-8174-63c8cc247b11 service nova] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:06:54.809559 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434f0e0d-de92-4747-8174-63c8cc247b11 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:06:54.813409 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-434f0e0d-de92-4747-8174-63c8cc247b11 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:06:54.961808 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434f0e0d-de92-4747-8174-63c8cc247b11 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:06:54.964626 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434f0e0d-de92-4747-8174-63c8cc247b11 service nova] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:06:54.993216 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 669/1368] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:06:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 749 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 20:06:56.314574 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c9093b-71c4-46c6-905b-0ef223dae68e None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:06:56.315949 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c9093b-71c4-46c6-905b-0ef223dae68e None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:06:56.316440 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c9093b-71c4-46c6-905b-0ef223dae68e None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:06:56.319649 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 700/1369] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:06:56 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:06:56.330225 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4346bfd6-37dd-49e6-b317-e098b69a0320 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:06:56.331050 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4346bfd6-37dd-49e6-b317-e098b69a0320 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:06:56.331194 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4346bfd6-37dd-49e6-b317-e098b69a0320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:06:56.397180 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:06:56.459789 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-4346bfd6-37dd-49e6-b317-e098b69a0320 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:06:56.461523 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-4346bfd6-37dd-49e6-b317-e098b69a0320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:06:56.665892 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 670/1370] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:06:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:06:57.126307 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c46f8ef-8479-4bd9-b3d4-ee4818f0b283 None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:06:57.126307 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c46f8ef-8479-4bd9-b3d4-ee4818f0b283 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:06:57.126307 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c46f8ef-8479-4bd9-b3d4-ee4818f0b283 None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:06:57.128955 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 701/1371] 10.222.0.20 () {54 vars in 964 bytes} [Tue Sep 6 20:06:57 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:06:57.134713 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc11bb5-1e98-4226-bfc6-92271df1c436 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:06:57.135301 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc11bb5-1e98-4226-bfc6-92271df1c436 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:06:57.136575 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc11bb5-1e98-4226-bfc6-92271df1c436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:06:57.296379 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-3fc11bb5-1e98-4226-bfc6-92271df1c436 None None] MFA Rules not processed for user `df1bca993bc142908cc7cf85eabd2b24`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:06:57.298567 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-3fc11bb5-1e98-4226-bfc6-92271df1c436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:06:57.372553 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:06:57.410932 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 671/1372] 10.222.0.20 () {58 vars in 1061 bytes} [Tue Sep 6 20:06:57 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:06:57.420075 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c46f8ef-8479-4bd9-b3d4-ee4818f0b283 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:06:57.421678 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-0c46f8ef-8479-4bd9-b3d4-ee4818f0b283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:06:57.566756 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c46f8ef-8479-4bd9-b3d4-ee4818f0b283 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:06:57.589652 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66d4cdf-0a7a-478f-9f75-0694e1f2444f None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:06:57.590058 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66d4cdf-0a7a-478f-9f75-0694e1f2444f None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:06:57.590470 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66d4cdf-0a7a-478f-9f75-0694e1f2444f None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:06:57.592293 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 672/1373] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:06:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 20:06:57.599460 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b93b84-926c-460b-a2d7-5d845447518d None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:06:57.599840 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b93b84-926c-460b-a2d7-5d845447518d None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:06:57.600430 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b93b84-926c-460b-a2d7-5d845447518d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:06:57.627133 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:06:57.677938 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52888a9d-8993-46e8-9107-cc1fcdba22cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df1bca993bc142908cc7cf85eabd2b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:06:57.678566 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52888a9d-8993-46e8-9107-cc1fcdba22cb service cinder] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:06:57.678851 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52888a9d-8993-46e8-9107-cc1fcdba22cb service cinder] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:06:57.679117 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52888a9d-8993-46e8-9107-cc1fcdba22cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:06:57.680830 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-52888a9d-8993-46e8-9107-cc1fcdba22cb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:06:57.687534 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-75b93b84-926c-460b-a2d7-5d845447518d None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:06:57.688772 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-75b93b84-926c-460b-a2d7-5d845447518d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:06:57.834817 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 673/1374] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:06:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:06:57.838613 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52888a9d-8993-46e8-9107-cc1fcdba22cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:06:57.840255 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52888a9d-8993-46e8-9107-cc1fcdba22cb service cinder] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:06:57.865026 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 702/1375] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:06:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 20:06:58.160355 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b93b84-926c-460b-a2d7-5d845447518d None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:06:58.161872 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-75b93b84-926c-460b-a2d7-5d845447518d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:06:58.240794 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:06:58.320386 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b93b84-926c-460b-a2d7-5d845447518d None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:06:58.481778 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a8629d-b6ce-4f29-baa5-9c9e55adc26f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5776cbf4f6714b2083c3ebdd6d98cc87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:06:58.482769 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a8629d-b6ce-4f29-baa5-9c9e55adc26f service nova] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:06:58.483197 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a8629d-b6ce-4f29-baa5-9c9e55adc26f service nova] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:06:58.483608 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a8629d-b6ce-4f29-baa5-9c9e55adc26f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:06:58.485706 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-84a8629d-b6ce-4f29-baa5-9c9e55adc26f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:06:58.567315 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:06:58.644053 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a8629d-b6ce-4f29-baa5-9c9e55adc26f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:06:58.647020 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a8629d-b6ce-4f29-baa5-9c9e55adc26f service nova] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:06:58.677483 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 674/1376] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:06:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 518 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 20:07:01.145802 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a493868-eb74-4355-9004-11766af401ec None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:07:01.145802 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a493868-eb74-4355-9004-11766af401ec None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:07:01.145802 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a493868-eb74-4355-9004-11766af401ec None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:07:01.147106 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 703/1377] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:07:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:07:01.155337 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a9c5f3-7fe7-40b7-97dc-678afd793c6f None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:07:01.155900 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a9c5f3-7fe7-40b7-97dc-678afd793c6f None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:07:01.156385 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a9c5f3-7fe7-40b7-97dc-678afd793c6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:07:01.272610 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-f6a9c5f3-7fe7-40b7-97dc-678afd793c6f None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:07:01.274839 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-f6a9c5f3-7fe7-40b7-97dc-678afd793c6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:07:01.429122 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 675/1378] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:07:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:07:01.770936 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a493868-eb74-4355-9004-11766af401ec None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:07:01.772561 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-6a493868-eb74-4355-9004-11766af401ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:07:01.920478 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:07:02.066403 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a493868-eb74-4355-9004-11766af401ec None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:07:02.292313 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03d122d-a940-4a16-b71b-277f335d11fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5776cbf4f6714b2083c3ebdd6d98cc87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:07:02.294159 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03d122d-a940-4a16-b71b-277f335d11fd service nova] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:07:02.294701 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03d122d-a940-4a16-b71b-277f335d11fd service nova] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:07:02.295039 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03d122d-a940-4a16-b71b-277f335d11fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:07:02.297101 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-f03d122d-a940-4a16-b71b-277f335d11fd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:07:02.342394 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:07:02.439060 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03d122d-a940-4a16-b71b-277f335d11fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:07:02.442632 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03d122d-a940-4a16-b71b-277f335d11fd service nova] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:07:02.476070 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 704/1379] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:07:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 706 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 20:07:05.705009 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964c24e6-f36a-421b-b9ec-375823e4ae6f None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:07:05.705805 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964c24e6-f36a-421b-b9ec-375823e4ae6f None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:07:05.705805 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964c24e6-f36a-421b-b9ec-375823e4ae6f None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:07:05.707405 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 676/1380] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:07:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:07:05.714729 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a358f340-8601-45a9-8c57-6941b5195380 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:07:05.715400 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a358f340-8601-45a9-8c57-6941b5195380 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:07:05.715400 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a358f340-8601-45a9-8c57-6941b5195380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:07:05.813202 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-a358f340-8601-45a9-8c57-6941b5195380 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:07:05.814956 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-a358f340-8601-45a9-8c57-6941b5195380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:07:05.999506 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 705/1381] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:07:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:07:06.341880 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964c24e6-f36a-421b-b9ec-375823e4ae6f None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:07:06.343559 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-964c24e6-f36a-421b-b9ec-375823e4ae6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:07:06.375350 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:07:06.567167 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964c24e6-f36a-421b-b9ec-375823e4ae6f None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:07:06.717666 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:07:06.743785 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3661a9-d7dd-42f6-b9d7-3228250e32c6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5776cbf4f6714b2083c3ebdd6d98cc87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:07:06.744421 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3661a9-d7dd-42f6-b9d7-3228250e32c6 service nova] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:07:06.744704 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3661a9-d7dd-42f6-b9d7-3228250e32c6 service nova] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:07:06.745516 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3661a9-d7dd-42f6-b9d7-3228250e32c6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:07:06.747600 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-5f3661a9-d7dd-42f6-b9d7-3228250e32c6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:07:06.925515 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3661a9-d7dd-42f6-b9d7-3228250e32c6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:07:06.928602 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3661a9-d7dd-42f6-b9d7-3228250e32c6 service nova] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:07:06.966787 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 677/1382] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:07:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 625 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:07:10.056291 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca6d43c-4c65-4b2c-a3ca-11309d967ddc None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:07:10.057288 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca6d43c-4c65-4b2c-a3ca-11309d967ddc None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:07:10.057933 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca6d43c-4c65-4b2c-a3ca-11309d967ddc None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:07:10.059380 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 706/1383] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:07:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:07:10.066919 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a55f70a-da45-46c3-ae43-21cf25796697 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:07:10.067719 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a55f70a-da45-46c3-ae43-21cf25796697 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:07:10.067945 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a55f70a-da45-46c3-ae43-21cf25796697 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:07:10.196892 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-3a55f70a-da45-46c3-ae43-21cf25796697 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:07:10.200362 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-3a55f70a-da45-46c3-ae43-21cf25796697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:07:10.326916 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:07:10.409431 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 678/1384] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:07:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 344 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:07:10.746565 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca6d43c-4c65-4b2c-a3ca-11309d967ddc None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:07:10.748385 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-5ca6d43c-4c65-4b2c-a3ca-11309d967ddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:07:10.760233 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:07:11.053209 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca6d43c-4c65-4b2c-a3ca-11309d967ddc None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:07:11.192734 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:07:11.263482 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cbc0b9-77be-436d-a652-5c548d584349 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5776cbf4f6714b2083c3ebdd6d98cc87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:07:11.264652 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cbc0b9-77be-436d-a652-5c548d584349 service nova] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:07:11.265077 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cbc0b9-77be-436d-a652-5c548d584349 service nova] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:07:11.265466 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cbc0b9-77be-436d-a652-5c548d584349 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:07:11.268181 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-76cbc0b9-77be-436d-a652-5c548d584349 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:07:11.426945 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cbc0b9-77be-436d-a652-5c548d584349 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:07:11.428646 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cbc0b9-77be-436d-a652-5c548d584349 service nova] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:07:11.453871 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 707/1385] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:07:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 708 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:07:13.958021 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd656705-91ba-40a0-a267-33bd06c5c89f None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:07:13.958637 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd656705-91ba-40a0-a267-33bd06c5c89f None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:07:13.958637 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd656705-91ba-40a0-a267-33bd06c5c89f None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:07:13.960076 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 679/1386] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:07:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 20:07:13.966746 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290a41e2-ef17-4f64-9ce6-7d7a1c6c1abc None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:07:13.967383 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290a41e2-ef17-4f64-9ce6-7d7a1c6c1abc None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:07:13.967491 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290a41e2-ef17-4f64-9ce6-7d7a1c6c1abc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:07:14.086353 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-290a41e2-ef17-4f64-9ce6-7d7a1c6c1abc None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:07:14.088389 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-290a41e2-ef17-4f64-9ce6-7d7a1c6c1abc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:07:14.189534 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:07:14.268566 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 708/1387] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:07:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:07:14.575722 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd656705-91ba-40a0-a267-33bd06c5c89f None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:07:14.577251 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-fd656705-91ba-40a0-a267-33bd06c5c89f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:07:14.772182 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd656705-91ba-40a0-a267-33bd06c5c89f None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:07:14.837826 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:07:14.942598 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b201dab6-b707-465c-b2fe-fe649b13aefb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5776cbf4f6714b2083c3ebdd6d98cc87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:07:14.943232 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b201dab6-b707-465c-b2fe-fe649b13aefb service nova] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:07:14.943574 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b201dab6-b707-465c-b2fe-fe649b13aefb service nova] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:07:14.944051 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b201dab6-b707-465c-b2fe-fe649b13aefb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:07:14.947367 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-b201dab6-b707-465c-b2fe-fe649b13aefb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:07:15.102400 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b201dab6-b707-465c-b2fe-fe649b13aefb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:07:15.106141 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b201dab6-b707-465c-b2fe-fe649b13aefb service nova] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:07:15.145344 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 680/1388] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:07:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 570 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:07:16.679606 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f75db1a-de1b-4a51-9da8-476198bcb025 None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:07:16.680339 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f75db1a-de1b-4a51-9da8-476198bcb025 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:07:16.680339 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f75db1a-de1b-4a51-9da8-476198bcb025 None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:07:16.682041 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 709/1389] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:07:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 20:07:16.689370 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78769f98-f4c4-4435-a7af-5336b6168877 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:07:16.690008 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78769f98-f4c4-4435-a7af-5336b6168877 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:07:16.690444 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78769f98-f4c4-4435-a7af-5336b6168877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:07:16.791000 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:07:16.808438 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-78769f98-f4c4-4435-a7af-5336b6168877 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:07:16.810720 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-78769f98-f4c4-4435-a7af-5336b6168877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:07:16.979146 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 681/1390] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:07:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:07:17.524317 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f66171-a47f-48c4-a313-3b4b5c4b3ab4 None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:07:17.524686 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f66171-a47f-48c4-a313-3b4b5c4b3ab4 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:07:17.525012 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f66171-a47f-48c4-a313-3b4b5c4b3ab4 None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:07:17.527136 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 710/1391] 10.222.0.20 () {54 vars in 961 bytes} [Tue Sep 6 20:07:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:07:17.536327 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb34ca4-c176-423a-9456-90b0a7997cb9 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:07:17.536816 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb34ca4-c176-423a-9456-90b0a7997cb9 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:07:17.537210 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb34ca4-c176-423a-9456-90b0a7997cb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:07:17.665490 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-1eb34ca4-c176-423a-9456-90b0a7997cb9 None None] MFA Rules not processed for user `07af18f35ffc42a295278d5349e06381`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:07:17.668305 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-1eb34ca4-c176-423a-9456-90b0a7997cb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:07:17.796179 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:07:17.807053 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 682/1392] 10.222.0.20 () {58 vars in 1058 bytes} [Tue Sep 6 20:07:17 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:07:17.818244 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f66171-a47f-48c4-a313-3b4b5c4b3ab4 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:07:17.820051 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-f2f66171-a47f-48c4-a313-3b4b5c4b3ab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:07:17.879186 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42d3c9a-9057-4199-952c-389b5782acf9 None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:07:17.879681 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42d3c9a-9057-4199-952c-389b5782acf9 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:07:17.880216 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42d3c9a-9057-4199-952c-389b5782acf9 None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:07:17.882067 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 683/1393] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:07:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 20:07:17.889098 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9263314f-3f8f-4a02-84ee-306c95f79d05 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:07:17.889450 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9263314f-3f8f-4a02-84ee-306c95f79d05 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:07:17.889934 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9263314f-3f8f-4a02-84ee-306c95f79d05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:07:17.981753 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f66171-a47f-48c4-a313-3b4b5c4b3ab4 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:07:17.990139 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-9263314f-3f8f-4a02-84ee-306c95f79d05 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:07:17.991493 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-9263314f-3f8f-4a02-84ee-306c95f79d05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:07:18.084481 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:07:18.086851 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95230c6b-1de1-4307-bc25-0ce95b5824ff service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07af18f35ffc42a295278d5349e06381', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:07:18.087554 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95230c6b-1de1-4307-bc25-0ce95b5824ff service glance] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:07:18.087859 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95230c6b-1de1-4307-bc25-0ce95b5824ff service glance] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:07:18.088188 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95230c6b-1de1-4307-bc25-0ce95b5824ff service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:07:18.091539 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-95230c6b-1de1-4307-bc25-0ce95b5824ff service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:07:18.171321 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 684/1394] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:07:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:07:18.257845 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95230c6b-1de1-4307-bc25-0ce95b5824ff service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:07:18.260598 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95230c6b-1de1-4307-bc25-0ce95b5824ff service glance] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:07:18.299719 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 711/1395] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:07:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:07:18.529147 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9263314f-3f8f-4a02-84ee-306c95f79d05 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:07:18.531195 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-9263314f-3f8f-4a02-84ee-306c95f79d05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:07:18.688377 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:07:18.755267 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9263314f-3f8f-4a02-84ee-306c95f79d05 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:07:18.952981 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd321d2-dcf4-4514-b23e-119ae0b3352e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5776cbf4f6714b2083c3ebdd6d98cc87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:07:18.953683 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd321d2-dcf4-4514-b23e-119ae0b3352e service nova] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:07:18.953909 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd321d2-dcf4-4514-b23e-119ae0b3352e service nova] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:07:18.954181 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd321d2-dcf4-4514-b23e-119ae0b3352e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:07:18.955866 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-0cd321d2-dcf4-4514-b23e-119ae0b3352e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:07:19.043781 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:07:19.067555 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd321d2-dcf4-4514-b23e-119ae0b3352e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:07:19.069196 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd321d2-dcf4-4514-b23e-119ae0b3352e service nova] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:07:19.092346 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 685/1396] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:07:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 564 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:07:19.439710 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fce5480-f6a4-4f07-8f17-8e3891a52918 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:07:19.440646 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fce5480-f6a4-4f07-8f17-8e3891a52918 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:07:19.440646 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fce5480-f6a4-4f07-8f17-8e3891a52918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:07:19.527824 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-4fce5480-f6a4-4f07-8f17-8e3891a52918 None None] MFA Rules not processed for user `1a52117df70447dba9bbee35d083e44f`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:07:19.529298 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-4fce5480-f6a4-4f07-8f17-8e3891a52918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:07:19.598876 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:07:19.646407 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 712/1397] 10.222.0.20 () {58 vars in 1009 bytes} [Tue Sep 6 20:07:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:07:19.707217 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28622b1-73ae-4cff-9097-776df4cfac21 None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:07:19.707635 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28622b1-73ae-4cff-9097-776df4cfac21 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:07:19.708007 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28622b1-73ae-4cff-9097-776df4cfac21 None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:07:19.709634 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 686/1398] 10.222.0.20 () {54 vars in 943 bytes} [Tue Sep 6 20:07:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:07:19.718246 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2134c077-e9d6-4e8e-b9d5-236d638e4dde None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:07:19.718674 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2134c077-e9d6-4e8e-b9d5-236d638e4dde None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:07:19.718970 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2134c077-e9d6-4e8e-b9d5-236d638e4dde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:07:19.864968 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-2134c077-e9d6-4e8e-b9d5-236d638e4dde None None] MFA Rules not processed for user `2b72516236da47e292d78de201b0552b`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:07:19.866445 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-2134c077-e9d6-4e8e-b9d5-236d638e4dde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:07:20.071537 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 713/1399] 10.222.0.20 () {58 vars in 1040 bytes} [Tue Sep 6 20:07:19 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 355 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:07:20.083050 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28622b1-73ae-4cff-9097-776df4cfac21 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:07:20.086005 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-b28622b1-73ae-4cff-9097-776df4cfac21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:07:20.377369 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28622b1-73ae-4cff-9097-776df4cfac21 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:07:20.454273 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:07:20.520236 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a231a67f-9962-46d0-afe5-d01feef47a3f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b72516236da47e292d78de201b0552b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:07:20.521037 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a231a67f-9962-46d0-afe5-d01feef47a3f service swift] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:07:20.521378 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a231a67f-9962-46d0-afe5-d01feef47a3f service swift] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:07:20.521773 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a231a67f-9962-46d0-afe5-d01feef47a3f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:07:20.523715 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-a231a67f-9962-46d0-afe5-d01feef47a3f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:07:20.636555 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a231a67f-9962-46d0-afe5-d01feef47a3f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:07:20.638208 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a231a67f-9962-46d0-afe5-d01feef47a3f service swift] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:07:20.651041 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 687/1400] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Sep 6 20:07:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 569 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 20:07:21.543175 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af422f4-f934-410d-bbb6-ab81437b08b2 None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:07:21.543999 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af422f4-f934-410d-bbb6-ab81437b08b2 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:07:21.544141 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af422f4-f934-410d-bbb6-ab81437b08b2 None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:07:21.546644 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 714/1401] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:07:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:07:21.553600 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3406d0-23b1-4c48-8ed9-00cc3b28c7f2 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:07:21.554423 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3406d0-23b1-4c48-8ed9-00cc3b28c7f2 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:07:21.554610 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3406d0-23b1-4c48-8ed9-00cc3b28c7f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:07:21.712263 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-2b3406d0-23b1-4c48-8ed9-00cc3b28c7f2 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:07:21.714878 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-2b3406d0-23b1-4c48-8ed9-00cc3b28c7f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:07:21.786153 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:07:21.897862 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 688/1402] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:07:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 346 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:07:22.230053 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af422f4-f934-410d-bbb6-ab81437b08b2 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:07:22.233017 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-0af422f4-f934-410d-bbb6-ab81437b08b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:07:22.296212 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:07:22.576338 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af422f4-f934-410d-bbb6-ab81437b08b2 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:07:22.801731 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:07:22.831062 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3322e924-dedd-4087-892b-9a5becb36df2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5776cbf4f6714b2083c3ebdd6d98cc87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:07:22.832130 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3322e924-dedd-4087-892b-9a5becb36df2 service nova] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:07:22.832625 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3322e924-dedd-4087-892b-9a5becb36df2 service nova] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:07:22.833106 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3322e924-dedd-4087-892b-9a5becb36df2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:07:22.836585 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-3322e924-dedd-4087-892b-9a5becb36df2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:07:23.035962 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3322e924-dedd-4087-892b-9a5becb36df2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:07:23.039416 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3322e924-dedd-4087-892b-9a5becb36df2 service nova] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:07:23.080509 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 715/1403] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:07:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 851 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:07:26.104241 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd36e637-eafa-41d0-9d17-38101dcc1c46 None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:07:26.104911 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd36e637-eafa-41d0-9d17-38101dcc1c46 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:07:26.105299 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd36e637-eafa-41d0-9d17-38101dcc1c46 None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:07:26.109765 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 689/1404] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:07:26 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:07:26.116637 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3242427-b4ce-4ca0-b22d-9dd44ad3fbe9 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:07:26.118162 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3242427-b4ce-4ca0-b22d-9dd44ad3fbe9 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:07:26.118550 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3242427-b4ce-4ca0-b22d-9dd44ad3fbe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:07:26.240459 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-b3242427-b4ce-4ca0-b22d-9dd44ad3fbe9 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:07:26.243411 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-b3242427-b4ce-4ca0-b22d-9dd44ad3fbe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:07:26.332285 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:07:26.392131 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 716/1405] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:07:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:07:26.947797 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd36e637-eafa-41d0-9d17-38101dcc1c46 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:07:26.950557 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-cd36e637-eafa-41d0-9d17-38101dcc1c46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:07:27.172316 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd36e637-eafa-41d0-9d17-38101dcc1c46 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:07:27.208085 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:07:27.327499 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e0968b-59e1-4e43-9db8-d1754934a9fb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5776cbf4f6714b2083c3ebdd6d98cc87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:07:27.329154 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e0968b-59e1-4e43-9db8-d1754934a9fb service nova] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:07:27.329433 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e0968b-59e1-4e43-9db8-d1754934a9fb service nova] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:07:27.329820 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e0968b-59e1-4e43-9db8-d1754934a9fb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:07:27.331566 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-42e0968b-59e1-4e43-9db8-d1754934a9fb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:07:27.484787 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e0968b-59e1-4e43-9db8-d1754934a9fb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:07:27.487343 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e0968b-59e1-4e43-9db8-d1754934a9fb service nova] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:07:27.517489 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 690/1406] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:07:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 570 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:07:30.240311 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c777bd0-2996-48d5-b163-6905e702ea2d None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:07:30.241320 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c777bd0-2996-48d5-b163-6905e702ea2d None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:07:30.241320 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c777bd0-2996-48d5-b163-6905e702ea2d None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:07:30.242579 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 717/1407] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:07:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 20:07:30.248762 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8b936d-48b3-43ed-b047-2986849d85a2 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:07:30.249809 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8b936d-48b3-43ed-b047-2986849d85a2 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:07:30.249809 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8b936d-48b3-43ed-b047-2986849d85a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:07:30.306796 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:07:30.362690 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-4d8b936d-48b3-43ed-b047-2986849d85a2 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:07:30.364927 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-4d8b936d-48b3-43ed-b047-2986849d85a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:07:30.544625 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 691/1408] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:07:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:07:30.900884 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c777bd0-2996-48d5-b163-6905e702ea2d None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:07:30.905424 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-8c777bd0-2996-48d5-b163-6905e702ea2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:07:31.157901 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c777bd0-2996-48d5-b163-6905e702ea2d None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:07:31.248284 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:07:31.389115 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bb7a1e-3f37-4381-89a8-15af3049e498 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5776cbf4f6714b2083c3ebdd6d98cc87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:07:31.391465 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bb7a1e-3f37-4381-89a8-15af3049e498 service nova] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:07:31.392542 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bb7a1e-3f37-4381-89a8-15af3049e498 service nova] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:07:31.393417 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bb7a1e-3f37-4381-89a8-15af3049e498 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:07:31.397450 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-02bb7a1e-3f37-4381-89a8-15af3049e498 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:07:31.584019 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bb7a1e-3f37-4381-89a8-15af3049e498 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:07:31.587965 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bb7a1e-3f37-4381-89a8-15af3049e498 service nova] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:07:31.611447 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 718/1409] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:07:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 712 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:07:40.541255 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=52 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:07:41.608094 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=77 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:07:47.992051 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42496835-e7f2-4b27-b188-f4a56337184e None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:07:47.992694 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42496835-e7f2-4b27-b188-f4a56337184e None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:07:47.992694 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42496835-e7f2-4b27-b188-f4a56337184e None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:07:47.994215 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 692/1410] 10.222.0.20 () {54 vars in 918 bytes} [Tue Sep 6 20:07:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:07:48.000747 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2cbe79-5d98-4619-84fb-e8e862d17254 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:07:48.001339 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2cbe79-5d98-4619-84fb-e8e862d17254 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:07:48.001339 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2cbe79-5d98-4619-84fb-e8e862d17254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:07:48.101840 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-1f2cbe79-5d98-4619-84fb-e8e862d17254 None None] MFA Rules not processed for user `f57872de7bb74ea48d30e93e51830aba`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:07:48.103160 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-1f2cbe79-5d98-4619-84fb-e8e862d17254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:07:48.233845 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 719/1411] 10.222.0.20 () {58 vars in 1015 bytes} [Tue Sep 6 20:07:47 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:07:48.251020 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062c3580-7603-4e7d-b2b7-907546ec8270 None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:07:48.251020 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062c3580-7603-4e7d-b2b7-907546ec8270 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:07:48.251338 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062c3580-7603-4e7d-b2b7-907546ec8270 None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:07:48.252945 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 693/1412] 10.222.0.20 () {54 vars in 918 bytes} [Tue Sep 6 20:07:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:07:48.258620 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aeb22b-3055-41ec-97f8-074aec74d6aa None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:07:48.258891 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aeb22b-3055-41ec-97f8-074aec74d6aa None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:07:48.259160 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aeb22b-3055-41ec-97f8-074aec74d6aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:07:48.346829 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:07:48.352951 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-f7aeb22b-3055-41ec-97f8-074aec74d6aa None None] MFA Rules not processed for user `5776cbf4f6714b2083c3ebdd6d98cc87`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:07:48.354625 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-f7aeb22b-3055-41ec-97f8-074aec74d6aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:07:48.503563 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 720/1413] 10.222.0.20 () {58 vars in 1015 bytes} [Tue Sep 6 20:07:48 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:07:51.273997 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed29611-9b0b-4f8c-b403-e6b6a677e18a None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:07:51.274868 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed29611-9b0b-4f8c-b403-e6b6a677e18a None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:07:51.274868 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed29611-9b0b-4f8c-b403-e6b6a677e18a None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:07:51.276569 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 694/1414] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 20:07:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 20:07:51.283699 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14771b66-0c56-47ef-8cef-df7cc1dac6c5 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:07:51.284517 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14771b66-0c56-47ef-8cef-df7cc1dac6c5 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:07:51.284517 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14771b66-0c56-47ef-8cef-df7cc1dac6c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:07:51.360240 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-14771b66-0c56-47ef-8cef-df7cc1dac6c5 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:07:51.361501 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-14771b66-0c56-47ef-8cef-df7cc1dac6c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:07:51.430466 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:07:51.501643 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 721/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 20:07:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:07:51.798360 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed29611-9b0b-4f8c-b403-e6b6a677e18a None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:07:51.801135 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-3ed29611-9b0b-4f8c-b403-e6b6a677e18a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:07:52.058768 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed29611-9b0b-4f8c-b403-e6b6a677e18a None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:07:52.177999 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:07:52.228316 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c389bb40-6783-48ab-af2e-25685e29c650 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5776cbf4f6714b2083c3ebdd6d98cc87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:07:52.228943 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c389bb40-6783-48ab-af2e-25685e29c650 service nova] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:07:52.229222 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c389bb40-6783-48ab-af2e-25685e29c650 service nova] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:07:52.229497 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c389bb40-6783-48ab-af2e-25685e29c650 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:07:52.231414 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-c389bb40-6783-48ab-af2e-25685e29c650 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:07:52.350523 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c389bb40-6783-48ab-af2e-25685e29c650 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:07:52.353760 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c389bb40-6783-48ab-af2e-25685e29c650 service nova] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:07:52.393102 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 695/1416] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:07:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 596 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:07:53.720465 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fd006d-df8a-4b2e-969d-1fc9fb86b455 None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:07:53.721061 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fd006d-df8a-4b2e-969d-1fc9fb86b455 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:07:53.721426 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fd006d-df8a-4b2e-969d-1fc9fb86b455 None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:07:53.724050 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 722/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 6 20:07:53 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:07:53.733570 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee22515f-55d9-42bc-9ad1-56e768cfccf7 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:07:53.734162 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee22515f-55d9-42bc-9ad1-56e768cfccf7 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:07:53.734272 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee22515f-55d9-42bc-9ad1-56e768cfccf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:07:53.841985 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-ee22515f-55d9-42bc-9ad1-56e768cfccf7 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:07:53.843247 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-ee22515f-55d9-42bc-9ad1-56e768cfccf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:07:53.876001 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:07:53.965492 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 696/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 6 20:07:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:07:54.130261 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fd006d-df8a-4b2e-969d-1fc9fb86b455 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:07:54.133005 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-22fd006d-df8a-4b2e-969d-1fc9fb86b455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:07:54.376437 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fd006d-df8a-4b2e-969d-1fc9fb86b455 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:07:54.428855 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:07:54.572412 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88bacd4-c807-4d37-89b0-f1097d7d1b0f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5776cbf4f6714b2083c3ebdd6d98cc87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:07:54.573713 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88bacd4-c807-4d37-89b0-f1097d7d1b0f service nova] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:07:54.574377 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88bacd4-c807-4d37-89b0-f1097d7d1b0f service nova] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:07:54.574896 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88bacd4-c807-4d37-89b0-f1097d7d1b0f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:07:54.578090 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-b88bacd4-c807-4d37-89b0-f1097d7d1b0f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:07:54.709781 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88bacd4-c807-4d37-89b0-f1097d7d1b0f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:07:54.711514 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88bacd4-c807-4d37-89b0-f1097d7d1b0f service nova] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:07:54.743364 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 723/1419] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:07:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 615 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:07:56.077889 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1440fa14-52d8-4d38-b4dc-fb3d2b7a09b6 None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:07:56.078537 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1440fa14-52d8-4d38-b4dc-fb3d2b7a09b6 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:07:56.078856 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1440fa14-52d8-4d38-b4dc-fb3d2b7a09b6 None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:07:56.081537 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 697/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 6 20:07:56 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:07:56.089252 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64841e59-7a58-4b62-bfd5-1ebd2ba03df9 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:07:56.090715 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64841e59-7a58-4b62-bfd5-1ebd2ba03df9 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:07:56.090715 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64841e59-7a58-4b62-bfd5-1ebd2ba03df9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:07:56.163570 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:07:56.193206 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-64841e59-7a58-4b62-bfd5-1ebd2ba03df9 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:07:56.194576 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-64841e59-7a58-4b62-bfd5-1ebd2ba03df9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:07:56.315391 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 724/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 6 20:07:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:07:56.480045 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1440fa14-52d8-4d38-b4dc-fb3d2b7a09b6 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:07:56.482848 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-1440fa14-52d8-4d38-b4dc-fb3d2b7a09b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:07:56.717383 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1440fa14-52d8-4d38-b4dc-fb3d2b7a09b6 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:07:56.735001 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:07:56.856538 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208126ed-ac99-4646-8375-a6a4506bfe93 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5776cbf4f6714b2083c3ebdd6d98cc87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:07:56.857337 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208126ed-ac99-4646-8375-a6a4506bfe93 service nova] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:07:56.859383 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208126ed-ac99-4646-8375-a6a4506bfe93 service nova] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:07:56.859383 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208126ed-ac99-4646-8375-a6a4506bfe93 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:07:56.860075 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-208126ed-ac99-4646-8375-a6a4506bfe93 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:07:56.998243 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208126ed-ac99-4646-8375-a6a4506bfe93 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:07:57.001301 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208126ed-ac99-4646-8375-a6a4506bfe93 service nova] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:07:57.037469 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 698/1422] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:07:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 558 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:07:58.370926 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2f0d20-dd96-494a-a203-dd0ae7a71ff9 None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:07:58.372332 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2f0d20-dd96-494a-a203-dd0ae7a71ff9 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:07:58.372332 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2f0d20-dd96-494a-a203-dd0ae7a71ff9 None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:07:58.374133 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 725/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 6 20:07:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:07:58.381644 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfa4711-548f-4b1f-88ac-bb22d4677539 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:07:58.382221 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfa4711-548f-4b1f-88ac-bb22d4677539 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:07:58.382369 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfa4711-548f-4b1f-88ac-bb22d4677539 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:07:58.407427 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:07:58.484823 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-4cfa4711-548f-4b1f-88ac-bb22d4677539 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:07:58.486297 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-4cfa4711-548f-4b1f-88ac-bb22d4677539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:07:58.667398 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 699/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 6 20:07:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:07:58.943865 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2f0d20-dd96-494a-a203-dd0ae7a71ff9 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:07:58.945473 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-1b2f0d20-dd96-494a-a203-dd0ae7a71ff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:07:59.089701 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:07:59.136223 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2f0d20-dd96-494a-a203-dd0ae7a71ff9 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:07:59.296311 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb11bed-c443-48c1-adff-87dc429b8940 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5776cbf4f6714b2083c3ebdd6d98cc87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:07:59.297077 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb11bed-c443-48c1-adff-87dc429b8940 service nova] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:07:59.297457 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb11bed-c443-48c1-adff-87dc429b8940 service nova] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:07:59.297987 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb11bed-c443-48c1-adff-87dc429b8940 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:07:59.300040 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-adb11bed-c443-48c1-adff-87dc429b8940 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:07:59.403750 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:07:59.429018 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb11bed-c443-48c1-adff-87dc429b8940 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:07:59.431948 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb11bed-c443-48c1-adff-87dc429b8940 service nova] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:07:59.461343 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 726/1425] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:07:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 519 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:08:00.681106 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a3b84e-ef22-448b-90bc-953ab2f9a829 None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:08:00.681998 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a3b84e-ef22-448b-90bc-953ab2f9a829 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:08:00.682176 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a3b84e-ef22-448b-90bc-953ab2f9a829 None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:08:00.684282 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 700/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 6 20:08:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:08:00.691803 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e39467e-d07d-49a8-95b7-7ec4026a839c None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:08:00.692350 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e39467e-d07d-49a8-95b7-7ec4026a839c None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:08:00.692350 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e39467e-d07d-49a8-95b7-7ec4026a839c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:08:00.781870 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-5e39467e-d07d-49a8-95b7-7ec4026a839c None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:08:00.783252 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-5e39467e-d07d-49a8-95b7-7ec4026a839c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:08:00.902434 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 727/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 6 20:08:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:08:01.167623 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a3b84e-ef22-448b-90bc-953ab2f9a829 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:08:01.169720 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-b2a3b84e-ef22-448b-90bc-953ab2f9a829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:08:01.279988 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:08:01.359224 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a3b84e-ef22-448b-90bc-953ab2f9a829 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:08:01.508461 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189bf05d-51a6-477c-8012-b8a46140bd62 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5776cbf4f6714b2083c3ebdd6d98cc87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:08:01.509316 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189bf05d-51a6-477c-8012-b8a46140bd62 service nova] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:08:01.509831 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189bf05d-51a6-477c-8012-b8a46140bd62 service nova] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:08:01.510323 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189bf05d-51a6-477c-8012-b8a46140bd62 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:08:01.513745 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-189bf05d-51a6-477c-8012-b8a46140bd62 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:08:01.604708 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:08:01.659173 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189bf05d-51a6-477c-8012-b8a46140bd62 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:08:01.660832 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189bf05d-51a6-477c-8012-b8a46140bd62 service nova] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:08:01.683655 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 701/1428] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:08:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 517 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:08:02.842722 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa3b48c-732f-4681-b85f-d684f4232c9e None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:08:02.843333 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa3b48c-732f-4681-b85f-d684f4232c9e None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:08:02.843333 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa3b48c-732f-4681-b85f-d684f4232c9e None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:08:02.845001 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 728/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 6 20:08:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:08:02.852349 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d6159d-184e-4f9f-8330-74e977b6ad97 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:08:02.852971 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d6159d-184e-4f9f-8330-74e977b6ad97 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:08:02.852971 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d6159d-184e-4f9f-8330-74e977b6ad97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:08:02.986410 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-b2d6159d-184e-4f9f-8330-74e977b6ad97 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:08:02.987763 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-b2d6159d-184e-4f9f-8330-74e977b6ad97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:08:03.130353 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 702/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 6 20:08:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:08:03.298662 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa3b48c-732f-4681-b85f-d684f4232c9e None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:08:03.300452 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-3aa3b48c-732f-4681-b85f-d684f4232c9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:08:03.364675 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:08:03.505508 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa3b48c-732f-4681-b85f-d684f4232c9e None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:08:03.659154 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9a0104-c3b3-4b99-8a1a-073bc5df5eff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5776cbf4f6714b2083c3ebdd6d98cc87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:08:03.659918 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9a0104-c3b3-4b99-8a1a-073bc5df5eff service nova] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:08:03.660265 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9a0104-c3b3-4b99-8a1a-073bc5df5eff service nova] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:08:03.660399 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9a0104-c3b3-4b99-8a1a-073bc5df5eff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:08:03.662190 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-4c9a0104-c3b3-4b99-8a1a-073bc5df5eff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:08:03.670883 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:08:03.797563 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9a0104-c3b3-4b99-8a1a-073bc5df5eff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:08:03.799571 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9a0104-c3b3-4b99-8a1a-073bc5df5eff service nova] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:08:03.824751 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 729/1431] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:08:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 527 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:08:05.058399 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703ce1c1-fdbe-4e75-b9cf-ec86b285a1bd None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:08:05.059622 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703ce1c1-fdbe-4e75-b9cf-ec86b285a1bd None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:08:05.059783 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703ce1c1-fdbe-4e75-b9cf-ec86b285a1bd None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:08:05.061085 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 703/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 6 20:08:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:08:05.068534 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d33ed66-65bc-4c09-996d-ed368375e2ff None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:08:05.069080 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d33ed66-65bc-4c09-996d-ed368375e2ff None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:08:05.069080 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d33ed66-65bc-4c09-996d-ed368375e2ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:08:05.157931 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-8d33ed66-65bc-4c09-996d-ed368375e2ff None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:08:05.159174 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-8d33ed66-65bc-4c09-996d-ed368375e2ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:08:05.240688 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:08:05.275364 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 730/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 6 20:08:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:08:05.432575 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703ce1c1-fdbe-4e75-b9cf-ec86b285a1bd None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:08:05.434562 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-703ce1c1-fdbe-4e75-b9cf-ec86b285a1bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:08:05.461926 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:08:05.686875 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703ce1c1-fdbe-4e75-b9cf-ec86b285a1bd None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:08:05.841347 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:08:05.875726 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16ff36b-5d02-4724-a8bd-81f7334f3631 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5776cbf4f6714b2083c3ebdd6d98cc87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:08:05.876442 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16ff36b-5d02-4724-a8bd-81f7334f3631 service nova] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:08:05.876762 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16ff36b-5d02-4724-a8bd-81f7334f3631 service nova] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:08:05.877077 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16ff36b-5d02-4724-a8bd-81f7334f3631 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:08:05.879009 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-e16ff36b-5d02-4724-a8bd-81f7334f3631 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:08:06.006587 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16ff36b-5d02-4724-a8bd-81f7334f3631 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:08:06.008158 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16ff36b-5d02-4724-a8bd-81f7334f3631 service nova] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:08:06.034218 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 704/1434] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:08:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 602 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 20:08:07.177832 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1b65be-fba7-424a-a83d-0be8c78ba1c7 None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:08:07.178466 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1b65be-fba7-424a-a83d-0be8c78ba1c7 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:08:07.178466 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1b65be-fba7-424a-a83d-0be8c78ba1c7 None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:08:07.180098 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 731/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 6 20:08:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 20:08:07.191508 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2bdf2d-c107-4e0c-a970-b9126da2bf49 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:08:07.192241 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2bdf2d-c107-4e0c-a970-b9126da2bf49 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:08:07.192451 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2bdf2d-c107-4e0c-a970-b9126da2bf49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:08:07.343207 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-5c2bdf2d-c107-4e0c-a970-b9126da2bf49 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:08:07.345248 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-5c2bdf2d-c107-4e0c-a970-b9126da2bf49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:08:07.428881 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:08:07.497149 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 705/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 6 20:08:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:08:07.657058 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1b65be-fba7-424a-a83d-0be8c78ba1c7 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:08:07.658769 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-cb1b65be-fba7-424a-a83d-0be8c78ba1c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:08:07.918209 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1b65be-fba7-424a-a83d-0be8c78ba1c7 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:08:07.990987 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:08:08.073148 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7372a95-e7c8-4f45-a76a-7e1eb7ccc8f0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5776cbf4f6714b2083c3ebdd6d98cc87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:08:08.073815 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7372a95-e7c8-4f45-a76a-7e1eb7ccc8f0 service nova] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:08:08.074095 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7372a95-e7c8-4f45-a76a-7e1eb7ccc8f0 service nova] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:08:08.074380 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7372a95-e7c8-4f45-a76a-7e1eb7ccc8f0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:08:08.076174 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-c7372a95-e7c8-4f45-a76a-7e1eb7ccc8f0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:08:08.188487 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7372a95-e7c8-4f45-a76a-7e1eb7ccc8f0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:08:08.190254 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7372a95-e7c8-4f45-a76a-7e1eb7ccc8f0 service nova] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:08:08.222682 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 732/1437] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:08:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 566 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 20:08:09.373319 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519f1b61-b0bf-46c1-b460-6785d85bc3ae None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:08:09.373988 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519f1b61-b0bf-46c1-b460-6785d85bc3ae None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:08:09.373988 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519f1b61-b0bf-46c1-b460-6785d85bc3ae None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:08:09.375611 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 706/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 6 20:08:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:08:09.384241 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfdded0-2183-442d-8803-aa3f38f0177a None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:08:09.384899 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfdded0-2183-442d-8803-aa3f38f0177a None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:08:09.384899 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfdded0-2183-442d-8803-aa3f38f0177a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:08:09.473985 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-3cfdded0-2183-442d-8803-aa3f38f0177a None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:08:09.475292 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-3cfdded0-2183-442d-8803-aa3f38f0177a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:08:09.477697 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:08:09.587914 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 733/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 6 20:08:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:08:09.745018 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519f1b61-b0bf-46c1-b460-6785d85bc3ae None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:08:09.746493 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-519f1b61-b0bf-46c1-b460-6785d85bc3ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:08:09.926915 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519f1b61-b0bf-46c1-b460-6785d85bc3ae None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:08:09.961812 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:08:10.084572 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92266f12-b8f8-4ccb-aed5-af3738cec8db service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5776cbf4f6714b2083c3ebdd6d98cc87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:08:10.085121 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92266f12-b8f8-4ccb-aed5-af3738cec8db service nova] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:08:10.085397 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92266f12-b8f8-4ccb-aed5-af3738cec8db service nova] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:08:10.085696 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92266f12-b8f8-4ccb-aed5-af3738cec8db service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:08:10.087485 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-92266f12-b8f8-4ccb-aed5-af3738cec8db service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:08:10.203033 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92266f12-b8f8-4ccb-aed5-af3738cec8db service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:08:10.204800 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92266f12-b8f8-4ccb-aed5-af3738cec8db service nova] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:08:10.226228 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 707/1440] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:08:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:08:12.554458 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46790778-4449-4228-a810-33544746de4e None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:08:12.555319 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46790778-4449-4228-a810-33544746de4e None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:08:12.555319 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46790778-4449-4228-a810-33544746de4e None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:08:12.556439 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 734/1441] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 20:08:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 20:08:12.567179 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58445b61-1f12-48e9-b3f6-e3ea1464b3fe None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:08:12.567797 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58445b61-1f12-48e9-b3f6-e3ea1464b3fe None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:08:12.568051 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58445b61-1f12-48e9-b3f6-e3ea1464b3fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:08:12.629735 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:08:12.666796 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-58445b61-1f12-48e9-b3f6-e3ea1464b3fe None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:08:12.668133 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-58445b61-1f12-48e9-b3f6-e3ea1464b3fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:08:12.920859 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 708/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 20:08:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 356 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:08:13.226469 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46790778-4449-4228-a810-33544746de4e None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:08:13.229552 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-46790778-4449-4228-a810-33544746de4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:08:13.366597 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46790778-4449-4228-a810-33544746de4e None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:08:13.405694 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:08:13.457239 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1933f6f8-dfd1-4f5d-a374-c8f70e61b48d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:08:13.458003 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1933f6f8-dfd1-4f5d-a374-c8f70e61b48d service neutron] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:08:13.458404 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1933f6f8-dfd1-4f5d-a374-c8f70e61b48d service neutron] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:08:13.458901 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1933f6f8-dfd1-4f5d-a374-c8f70e61b48d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:08:13.461089 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-1933f6f8-dfd1-4f5d-a374-c8f70e61b48d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:08:13.593683 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1933f6f8-dfd1-4f5d-a374-c8f70e61b48d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:08:13.595378 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1933f6f8-dfd1-4f5d-a374-c8f70e61b48d service neutron] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:08:13.623184 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 735/1443] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:08:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:08:16.234619 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b09099-35ba-4189-afe1-0d8cbc076dd3 None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:08:16.235659 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b09099-35ba-4189-afe1-0d8cbc076dd3 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:08:16.235659 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b09099-35ba-4189-afe1-0d8cbc076dd3 None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:08:16.236976 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 709/1444] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 20:08:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:08:16.243372 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e509f0-5ccc-4e6e-9b0b-32082862cc73 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:08:16.244255 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e509f0-5ccc-4e6e-9b0b-32082862cc73 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:08:16.244255 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e509f0-5ccc-4e6e-9b0b-32082862cc73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:08:16.325806 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-67e509f0-5ccc-4e6e-9b0b-32082862cc73 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:08:16.327367 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-67e509f0-5ccc-4e6e-9b0b-32082862cc73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:08:16.343547 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:08:16.452568 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 736/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 20:08:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:08:16.744299 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b09099-35ba-4189-afe1-0d8cbc076dd3 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:08:16.745962 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-a0b09099-35ba-4189-afe1-0d8cbc076dd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:08:16.883450 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b09099-35ba-4189-afe1-0d8cbc076dd3 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:08:16.901311 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:08:17.013412 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56dedde-df3d-47b5-9e2f-f5df8ecacd67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:08:17.014438 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56dedde-df3d-47b5-9e2f-f5df8ecacd67 service neutron] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:08:17.014841 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56dedde-df3d-47b5-9e2f-f5df8ecacd67 service neutron] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:08:17.015331 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56dedde-df3d-47b5-9e2f-f5df8ecacd67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:08:17.018291 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-a56dedde-df3d-47b5-9e2f-f5df8ecacd67 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:08:17.201101 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56dedde-df3d-47b5-9e2f-f5df8ecacd67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:08:17.202590 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56dedde-df3d-47b5-9e2f-f5df8ecacd67 service neutron] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:08:17.225827 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 710/1446] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:08:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 20:08:20.036972 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dea7182-adf3-4a98-aeb0-ca19ea22abb0 None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:08:20.037589 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dea7182-adf3-4a98-aeb0-ca19ea22abb0 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:08:20.037971 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dea7182-adf3-4a98-aeb0-ca19ea22abb0 None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:08:20.040477 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 737/1447] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 20:08:20 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:08:20.047073 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab2d6d4-939f-48e9-8a0a-0325db1ed5fd None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:08:20.047765 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab2d6d4-939f-48e9-8a0a-0325db1ed5fd None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:08:20.047765 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab2d6d4-939f-48e9-8a0a-0325db1ed5fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:08:20.109595 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:08:20.133135 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-eab2d6d4-939f-48e9-8a0a-0325db1ed5fd None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:08:20.134680 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-eab2d6d4-939f-48e9-8a0a-0325db1ed5fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:08:20.259018 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 711/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 20:08:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:08:20.554995 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dea7182-adf3-4a98-aeb0-ca19ea22abb0 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:08:20.557145 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-7dea7182-adf3-4a98-aeb0-ca19ea22abb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:08:20.691675 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dea7182-adf3-4a98-aeb0-ca19ea22abb0 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:08:20.742671 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:08:20.785688 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198419cd-70c0-41ad-a95f-c4c42cf95fe8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:08:20.786311 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198419cd-70c0-41ad-a95f-c4c42cf95fe8 service neutron] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:08:20.786593 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198419cd-70c0-41ad-a95f-c4c42cf95fe8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:08:20.786866 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198419cd-70c0-41ad-a95f-c4c42cf95fe8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:08:20.788550 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-198419cd-70c0-41ad-a95f-c4c42cf95fe8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:08:20.932236 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198419cd-70c0-41ad-a95f-c4c42cf95fe8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:08:20.934081 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198419cd-70c0-41ad-a95f-c4c42cf95fe8 service neutron] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:08:20.957037 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 738/1449] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:08:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 20:08:23.560578 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f33f330-2bee-4fe1-a271-9927f2045436 None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:08:23.561116 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f33f330-2bee-4fe1-a271-9927f2045436 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:08:23.561417 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f33f330-2bee-4fe1-a271-9927f2045436 None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:08:23.564067 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 712/1450] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 20:08:23 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:08:23.572112 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1d58db-e2b0-4362-a103-7c62f5445af3 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:08:23.572946 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1d58db-e2b0-4362-a103-7c62f5445af3 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:08:23.572946 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1d58db-e2b0-4362-a103-7c62f5445af3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:08:23.671397 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-9c1d58db-e2b0-4362-a103-7c62f5445af3 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:08:23.672867 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-9c1d58db-e2b0-4362-a103-7c62f5445af3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:08:23.701083 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:08:23.806439 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 739/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 20:08:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:08:24.129289 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f33f330-2bee-4fe1-a271-9927f2045436 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:08:24.130985 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-0f33f330-2bee-4fe1-a271-9927f2045436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:08:24.310626 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f33f330-2bee-4fe1-a271-9927f2045436 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:08:24.335860 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:08:24.399100 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca85cfb-c5b5-48e1-84c5-150f4f2f7d10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:08:24.399633 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca85cfb-c5b5-48e1-84c5-150f4f2f7d10 service neutron] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:08:24.399912 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca85cfb-c5b5-48e1-84c5-150f4f2f7d10 service neutron] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:08:24.400183 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca85cfb-c5b5-48e1-84c5-150f4f2f7d10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:08:24.401940 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-fca85cfb-c5b5-48e1-84c5-150f4f2f7d10 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:08:24.534435 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca85cfb-c5b5-48e1-84c5-150f4f2f7d10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:08:24.535891 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca85cfb-c5b5-48e1-84c5-150f4f2f7d10 service neutron] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:08:24.556718 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 713/1452] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:08:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:08:26.999644 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0727d55a-9674-476a-a096-c7d09bc6918d None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:08:27.000392 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0727d55a-9674-476a-a096-c7d09bc6918d None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:08:27.000392 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0727d55a-9674-476a-a096-c7d09bc6918d None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:08:27.002194 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 740/1453] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 20:08:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:08:27.008987 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4ca330-7008-4420-a092-85ffb184d5d9 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:08:27.009961 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4ca330-7008-4420-a092-85ffb184d5d9 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:08:27.009961 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4ca330-7008-4420-a092-85ffb184d5d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:08:27.090931 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:08:27.096318 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-2e4ca330-7008-4420-a092-85ffb184d5d9 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:08:27.098421 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-2e4ca330-7008-4420-a092-85ffb184d5d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:08:27.247142 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 714/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 20:08:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:08:27.563671 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0727d55a-9674-476a-a096-c7d09bc6918d None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:08:27.565435 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-0727d55a-9674-476a-a096-c7d09bc6918d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:08:27.730868 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0727d55a-9674-476a-a096-c7d09bc6918d None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:08:27.797337 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:08:27.829785 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97715df6-fc3b-41ab-870e-2462fa88a0de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:08:27.830555 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97715df6-fc3b-41ab-870e-2462fa88a0de service neutron] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:08:27.830931 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97715df6-fc3b-41ab-870e-2462fa88a0de service neutron] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:08:27.831293 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97715df6-fc3b-41ab-870e-2462fa88a0de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:08:27.833253 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-97715df6-fc3b-41ab-870e-2462fa88a0de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:08:28.024236 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97715df6-fc3b-41ab-870e-2462fa88a0de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:08:28.026547 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97715df6-fc3b-41ab-870e-2462fa88a0de service neutron] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:08:28.072212 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 741/1455] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:08:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 509 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:08:30.691674 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5dff24-b0c0-4cc8-a02b-467afb42f9a0 None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:08:30.692559 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5dff24-b0c0-4cc8-a02b-467afb42f9a0 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:08:30.692559 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5dff24-b0c0-4cc8-a02b-467afb42f9a0 None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:08:30.693787 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 715/1456] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 20:08:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:08:30.700377 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e005b7a7-5c16-4dbe-8fe7-b5407287781c None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:08:30.701036 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e005b7a7-5c16-4dbe-8fe7-b5407287781c None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:08:30.701036 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e005b7a7-5c16-4dbe-8fe7-b5407287781c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:08:30.783549 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-e005b7a7-5c16-4dbe-8fe7-b5407287781c None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:08:30.785176 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-e005b7a7-5c16-4dbe-8fe7-b5407287781c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:08:30.823903 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:08:30.921818 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 742/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 20:08:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:08:31.228073 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5dff24-b0c0-4cc8-a02b-467afb42f9a0 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:08:31.229742 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-8f5dff24-b0c0-4cc8-a02b-467afb42f9a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:08:31.360164 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5dff24-b0c0-4cc8-a02b-467afb42f9a0 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:08:31.395630 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:08:31.455874 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe70137-047a-4630-b8e0-ee4bb9d81ea5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:08:31.456661 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe70137-047a-4630-b8e0-ee4bb9d81ea5 service neutron] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:08:31.456940 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe70137-047a-4630-b8e0-ee4bb9d81ea5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:08:31.457256 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe70137-047a-4630-b8e0-ee4bb9d81ea5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:08:31.459098 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-4fe70137-047a-4630-b8e0-ee4bb9d81ea5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:08:31.604341 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe70137-047a-4630-b8e0-ee4bb9d81ea5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:08:31.605907 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe70137-047a-4630-b8e0-ee4bb9d81ea5 service neutron] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:08:31.628067 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 716/1458] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:08:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:08:34.217825 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea684ae3-c3ea-4608-a83a-1a15e4ed9eee None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:08:34.218535 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea684ae3-c3ea-4608-a83a-1a15e4ed9eee None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:08:34.218535 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea684ae3-c3ea-4608-a83a-1a15e4ed9eee None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:08:34.220006 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 743/1459] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 20:08:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:08:34.226592 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5344be0-5958-41c2-9348-a01d581ddd41 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:08:34.227191 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5344be0-5958-41c2-9348-a01d581ddd41 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:08:34.227191 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5344be0-5958-41c2-9348-a01d581ddd41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:08:34.314837 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-f5344be0-5958-41c2-9348-a01d581ddd41 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:08:34.316207 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-f5344be0-5958-41c2-9348-a01d581ddd41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:08:34.327060 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:08:34.447953 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 717/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 20:08:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:08:34.768675 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea684ae3-c3ea-4608-a83a-1a15e4ed9eee None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:08:34.770586 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-ea684ae3-c3ea-4608-a83a-1a15e4ed9eee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:08:34.914511 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea684ae3-c3ea-4608-a83a-1a15e4ed9eee None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:08:35.020088 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:08:35.058053 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7004be29-1e9e-4ef0-b0fc-f5e7909c050a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:08:35.059080 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7004be29-1e9e-4ef0-b0fc-f5e7909c050a service neutron] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:08:35.059578 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7004be29-1e9e-4ef0-b0fc-f5e7909c050a service neutron] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:08:35.060176 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7004be29-1e9e-4ef0-b0fc-f5e7909c050a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:08:35.063970 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-7004be29-1e9e-4ef0-b0fc-f5e7909c050a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:08:35.282914 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7004be29-1e9e-4ef0-b0fc-f5e7909c050a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:08:35.285590 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7004be29-1e9e-4ef0-b0fc-f5e7909c050a service neutron] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:08:35.322221 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 744/1461] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:08:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 554 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:08:37.842498 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2729ca5-ec75-443a-bf11-ff58f6297ed2 None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:08:37.843206 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2729ca5-ec75-443a-bf11-ff58f6297ed2 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:08:37.843206 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2729ca5-ec75-443a-bf11-ff58f6297ed2 None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:08:37.844663 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 718/1462] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 20:08:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:08:37.851215 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b87ec1-23e8-47b0-9da4-39e0a3210854 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:08:37.851815 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b87ec1-23e8-47b0-9da4-39e0a3210854 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:08:37.851815 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b87ec1-23e8-47b0-9da4-39e0a3210854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:08:37.947073 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-51b87ec1-23e8-47b0-9da4-39e0a3210854 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:08:37.948426 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-51b87ec1-23e8-47b0-9da4-39e0a3210854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:08:37.998897 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:08:38.079284 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 745/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 20:08:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:08:38.379876 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2729ca5-ec75-443a-bf11-ff58f6297ed2 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:08:38.381583 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-d2729ca5-ec75-443a-bf11-ff58f6297ed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:08:38.533876 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2729ca5-ec75-443a-bf11-ff58f6297ed2 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:08:38.625470 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:08:38.678772 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e2b82f-0af5-4534-a91a-7dc91074680b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:08:38.679532 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e2b82f-0af5-4534-a91a-7dc91074680b service neutron] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:08:38.679913 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e2b82f-0af5-4534-a91a-7dc91074680b service neutron] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:08:38.680284 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e2b82f-0af5-4534-a91a-7dc91074680b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:08:38.682399 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-53e2b82f-0af5-4534-a91a-7dc91074680b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:08:38.846109 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e2b82f-0af5-4534-a91a-7dc91074680b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:08:38.849969 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e2b82f-0af5-4534-a91a-7dc91074680b service neutron] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:08:38.877313 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 719/1464] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:08:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 498 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:08:41.299714 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7656c011-4ce7-4103-9da8-cc7d62ca3896 None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:08:41.300327 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7656c011-4ce7-4103-9da8-cc7d62ca3896 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:08:41.300327 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7656c011-4ce7-4103-9da8-cc7d62ca3896 None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:08:41.301729 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 746/1465] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 20:08:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:08:41.308006 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966015cd-ea14-41df-b79b-36df6fc4280b None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:08:41.308555 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966015cd-ea14-41df-b79b-36df6fc4280b None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:08:41.308555 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966015cd-ea14-41df-b79b-36df6fc4280b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:08:41.390953 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-966015cd-ea14-41df-b79b-36df6fc4280b None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:08:41.392206 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-966015cd-ea14-41df-b79b-36df6fc4280b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:08:41.415021 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:08:41.510045 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 720/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 20:08:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:08:41.821483 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7656c011-4ce7-4103-9da8-cc7d62ca3896 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:08:41.823134 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-7656c011-4ce7-4103-9da8-cc7d62ca3896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:08:41.979883 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7656c011-4ce7-4103-9da8-cc7d62ca3896 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:08:42.084894 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:08:42.097999 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b623a24-94a7-4514-9e91-6173cd121c78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:08:42.099044 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b623a24-94a7-4514-9e91-6173cd121c78 service neutron] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:08:42.099579 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b623a24-94a7-4514-9e91-6173cd121c78 service neutron] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:08:42.100054 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b623a24-94a7-4514-9e91-6173cd121c78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:08:42.103344 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-3b623a24-94a7-4514-9e91-6173cd121c78 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:08:42.284320 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b623a24-94a7-4514-9e91-6173cd121c78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:08:42.286404 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b623a24-94a7-4514-9e91-6173cd121c78 service neutron] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:08:42.310088 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 747/1467] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:08:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 489 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:08:45.294431 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c050bf9e-acf6-482f-8459-e64c5b393d02 None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:08:45.295011 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c050bf9e-acf6-482f-8459-e64c5b393d02 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:08:45.296224 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c050bf9e-acf6-482f-8459-e64c5b393d02 None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:08:45.301426 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 721/1468] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 20:08:45 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:08:45.308853 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83479566-e09c-42db-9402-44af7fc03e9a None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:08:45.309674 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83479566-e09c-42db-9402-44af7fc03e9a None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:08:45.309882 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83479566-e09c-42db-9402-44af7fc03e9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:08:45.392548 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-83479566-e09c-42db-9402-44af7fc03e9a None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:08:45.394130 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-83479566-e09c-42db-9402-44af7fc03e9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:08:45.454651 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:08:45.541587 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 748/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 20:08:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:08:45.850943 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c050bf9e-acf6-482f-8459-e64c5b393d02 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:08:45.852648 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-c050bf9e-acf6-482f-8459-e64c5b393d02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:08:45.985297 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c050bf9e-acf6-482f-8459-e64c5b393d02 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:08:46.046246 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:08:46.096579 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490ee518-3e61-4a9d-b003-f2cffba90c1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:08:46.097503 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490ee518-3e61-4a9d-b003-f2cffba90c1c service neutron] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:08:46.098018 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490ee518-3e61-4a9d-b003-f2cffba90c1c service neutron] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:08:46.098483 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490ee518-3e61-4a9d-b003-f2cffba90c1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:08:46.101490 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-490ee518-3e61-4a9d-b003-f2cffba90c1c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:08:46.268842 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490ee518-3e61-4a9d-b003-f2cffba90c1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:08:46.270479 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490ee518-3e61-4a9d-b003-f2cffba90c1c service neutron] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:08:46.296957 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 722/1470] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:08:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:08:55.539607 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=22 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:08:56.292385 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=66 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:09:09.913744 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e913c1-bbe3-48fb-936c-e6134259c931 None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:09:09.914583 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e913c1-bbe3-48fb-936c-e6134259c931 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:09:09.914583 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e913c1-bbe3-48fb-936c-e6134259c931 None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:09:09.915832 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 749/1471] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:09:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:09:09.922608 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0dfe77-f35e-42ec-9e18-45301b192885 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:09:09.923396 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0dfe77-f35e-42ec-9e18-45301b192885 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:09:09.923396 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0dfe77-f35e-42ec-9e18-45301b192885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:09:10.003847 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-1f0dfe77-f35e-42ec-9e18-45301b192885 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:09:10.005143 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-1f0dfe77-f35e-42ec-9e18-45301b192885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:09:10.143021 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 723/1472] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:09:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:09:10.467623 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096c60b0-9bfd-4061-b2b8-42c4794e65d5 None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:09:10.467893 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096c60b0-9bfd-4061-b2b8-42c4794e65d5 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:09:10.468308 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096c60b0-9bfd-4061-b2b8-42c4794e65d5 None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:09:10.470391 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 750/1473] 10.222.0.20 () {54 vars in 961 bytes} [Tue Sep 6 20:09:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:09:10.479037 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a632c08-bc31-4860-b20b-5dfd191a6fc6 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:09:10.479680 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a632c08-bc31-4860-b20b-5dfd191a6fc6 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:09:10.479980 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a632c08-bc31-4860-b20b-5dfd191a6fc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:09:10.583035 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-4a632c08-bc31-4860-b20b-5dfd191a6fc6 None None] MFA Rules not processed for user `07af18f35ffc42a295278d5349e06381`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:09:10.584344 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-4a632c08-bc31-4860-b20b-5dfd191a6fc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:09:10.612395 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:09:10.686147 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 724/1474] 10.222.0.20 () {58 vars in 1058 bytes} [Tue Sep 6 20:09:10 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:09:10.694758 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096c60b0-9bfd-4061-b2b8-42c4794e65d5 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:09:10.696361 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-096c60b0-9bfd-4061-b2b8-42c4794e65d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:09:10.830088 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096c60b0-9bfd-4061-b2b8-42c4794e65d5 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:09:10.942372 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf7ffd0-1a92-413d-8e6d-8865f04014c9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07af18f35ffc42a295278d5349e06381', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:09:10.943176 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf7ffd0-1a92-413d-8e6d-8865f04014c9 service glance] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:09:10.943537 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf7ffd0-1a92-413d-8e6d-8865f04014c9 service glance] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:09:10.944008 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf7ffd0-1a92-413d-8e6d-8865f04014c9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:09:10.947360 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-aaf7ffd0-1a92-413d-8e6d-8865f04014c9 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:09:11.045214 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:09:11.134845 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf7ffd0-1a92-413d-8e6d-8865f04014c9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:09:11.136575 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf7ffd0-1a92-413d-8e6d-8865f04014c9 service glance] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:09:11.161055 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 751/1475] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:09:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 20:09:13.729471 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ad3408-ed01-4a2b-82ae-8515b9d65518 None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:09:13.730283 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ad3408-ed01-4a2b-82ae-8515b9d65518 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:09:13.730464 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ad3408-ed01-4a2b-82ae-8515b9d65518 None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:09:13.731843 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 725/1476] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:09:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 20:09:13.741745 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1050c7-7f6d-44c1-bc76-9962f5b53da4 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:09:13.742594 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1050c7-7f6d-44c1-bc76-9962f5b53da4 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:09:13.742769 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1050c7-7f6d-44c1-bc76-9962f5b53da4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:09:13.826756 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-6b1050c7-7f6d-44c1-bc76-9962f5b53da4 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:09:13.827962 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-6b1050c7-7f6d-44c1-bc76-9962f5b53da4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:09:13.951995 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 752/1477] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:09:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:09:14.241970 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ad3408-ed01-4a2b-82ae-8515b9d65518 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:09:14.243935 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-36ad3408-ed01-4a2b-82ae-8515b9d65518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:09:14.406338 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ad3408-ed01-4a2b-82ae-8515b9d65518 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:09:14.467256 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:09:14.532730 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fe6af0-efa6-4eb8-9ca2-d2fadb82e68c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:09:14.533370 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fe6af0-efa6-4eb8-9ca2-d2fadb82e68c admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:09:14.533690 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fe6af0-efa6-4eb8-9ca2-d2fadb82e68c admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:09:14.533964 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fe6af0-efa6-4eb8-9ca2-d2fadb82e68c admin admin] PATH_INFO: `/v3/projects` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:09:14.534636 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fe6af0-efa6-4eb8-9ca2-d2fadb82e68c admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:09:14.536766 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: WARNING py.warnings [None req-61fe6af0-efa6-4eb8-9ca2-d2fadb82e68c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 06 20:09:14.536766 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: warnings.warn(msg) Sep 06 20:09:14.536766 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: Sep 06 20:09:14.537977 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fe6af0-efa6-4eb8-9ca2-d2fadb82e68c admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:09:14.563357 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 726/1478] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 6 20:09:14 2022] GET /identity/v3/projects => generated 2553 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:09:16.943780 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72590f7-807f-4449-a125-32854f9a1d4b None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:09:16.944644 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72590f7-807f-4449-a125-32854f9a1d4b None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:09:16.944644 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72590f7-807f-4449-a125-32854f9a1d4b None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:09:16.945859 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 753/1479] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:09:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:09:16.952512 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1eb600-1957-4da7-b2c6-1e2fa3233acf None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:09:16.953907 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1eb600-1957-4da7-b2c6-1e2fa3233acf None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:09:16.954069 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1eb600-1957-4da7-b2c6-1e2fa3233acf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:09:17.037944 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-ef1eb600-1957-4da7-b2c6-1e2fa3233acf None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:09:17.039469 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-ef1eb600-1957-4da7-b2c6-1e2fa3233acf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:09:17.163970 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 727/1480] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:09:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:09:17.616260 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72590f7-807f-4449-a125-32854f9a1d4b None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:09:17.618139 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-a72590f7-807f-4449-a125-32854f9a1d4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:09:17.630341 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:09:17.853719 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72590f7-807f-4449-a125-32854f9a1d4b None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:09:18.017296 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:09:18.077469 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb64b62c-5608-41c5-80f5-2ac6e672b375 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5776cbf4f6714b2083c3ebdd6d98cc87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:09:18.078418 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb64b62c-5608-41c5-80f5-2ac6e672b375 service nova] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:09:18.078710 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb64b62c-5608-41c5-80f5-2ac6e672b375 service nova] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:09:18.079012 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb64b62c-5608-41c5-80f5-2ac6e672b375 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:09:18.080893 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-cb64b62c-5608-41c5-80f5-2ac6e672b375 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:09:18.242544 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb64b62c-5608-41c5-80f5-2ac6e672b375 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:09:18.244410 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb64b62c-5608-41c5-80f5-2ac6e672b375 service nova] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:09:18.269184 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 754/1481] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:09:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 656 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:09:20.932712 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740c0742-4bf7-493e-8881-16c7c6db23bc None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:09:20.933598 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740c0742-4bf7-493e-8881-16c7c6db23bc None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:09:20.933598 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740c0742-4bf7-493e-8881-16c7c6db23bc None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:09:20.935243 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 728/1482] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:09:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:09:20.942186 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e28323-68ee-4d58-afcb-1a6380f266cd None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:09:20.942786 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e28323-68ee-4d58-afcb-1a6380f266cd None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:09:20.942786 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e28323-68ee-4d58-afcb-1a6380f266cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:09:21.022740 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-a3e28323-68ee-4d58-afcb-1a6380f266cd None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:09:21.024007 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-a3e28323-68ee-4d58-afcb-1a6380f266cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:09:21.080736 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:09:21.143763 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 755/1483] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:09:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:09:21.448640 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740c0742-4bf7-493e-8881-16c7c6db23bc None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:09:21.450250 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-740c0742-4bf7-493e-8881-16c7c6db23bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:09:21.480401 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:09:21.623727 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740c0742-4bf7-493e-8881-16c7c6db23bc None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:09:21.743696 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502ca28b-a8d2-4dd5-8d5f-863d93b4dca9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:09:21.744699 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502ca28b-a8d2-4dd5-8d5f-863d93b4dca9 service neutron] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:09:21.745168 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502ca28b-a8d2-4dd5-8d5f-863d93b4dca9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:09:21.745710 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502ca28b-a8d2-4dd5-8d5f-863d93b4dca9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:09:21.748995 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-502ca28b-a8d2-4dd5-8d5f-863d93b4dca9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:09:21.900537 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:09:21.949547 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502ca28b-a8d2-4dd5-8d5f-863d93b4dca9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:09:21.951463 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502ca28b-a8d2-4dd5-8d5f-863d93b4dca9 service neutron] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:09:21.976480 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 729/1484] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:09:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 528 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:09:25.355363 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f529d4b-5e94-4943-a913-ba0a9aab3bde None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:09:25.356127 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f529d4b-5e94-4943-a913-ba0a9aab3bde None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:09:25.356127 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f529d4b-5e94-4943-a913-ba0a9aab3bde None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:09:25.357656 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 756/1485] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:09:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:09:25.364956 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc1e16b-1c4b-444e-8202-5abad06999d1 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:09:25.365684 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc1e16b-1c4b-444e-8202-5abad06999d1 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:09:25.366053 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc1e16b-1c4b-444e-8202-5abad06999d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:09:25.497657 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-5bc1e16b-1c4b-444e-8202-5abad06999d1 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:09:25.499931 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-5bc1e16b-1c4b-444e-8202-5abad06999d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:09:25.636857 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 730/1486] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:09:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:09:25.949961 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f529d4b-5e94-4943-a913-ba0a9aab3bde None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:09:25.951583 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-4f529d4b-5e94-4943-a913-ba0a9aab3bde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:09:26.095182 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f529d4b-5e94-4943-a913-ba0a9aab3bde None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:09:26.186616 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:09:26.190180 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332cacb3-1148-4a45-a77b-93691657174a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:09:26.191193 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332cacb3-1148-4a45-a77b-93691657174a service neutron] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:09:26.191743 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332cacb3-1148-4a45-a77b-93691657174a service neutron] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:09:26.192165 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332cacb3-1148-4a45-a77b-93691657174a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:09:26.194335 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-332cacb3-1148-4a45-a77b-93691657174a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:09:26.340815 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332cacb3-1148-4a45-a77b-93691657174a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:09:26.342360 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332cacb3-1148-4a45-a77b-93691657174a service neutron] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:09:26.364098 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 757/1487] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:09:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 20:09:29.126517 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f981cb-bae0-4f09-831b-7477a704418f None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:09:29.127108 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f981cb-bae0-4f09-831b-7477a704418f None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:09:29.127447 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f981cb-bae0-4f09-831b-7477a704418f None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:09:29.130053 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 731/1488] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:09:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:09:29.136672 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6377c2e4-eb47-4fb4-9ea6-648bad7f5343 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:09:29.137223 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6377c2e4-eb47-4fb4-9ea6-648bad7f5343 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:09:29.137223 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6377c2e4-eb47-4fb4-9ea6-648bad7f5343 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:09:29.223400 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-6377c2e4-eb47-4fb4-9ea6-648bad7f5343 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:09:29.224728 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-6377c2e4-eb47-4fb4-9ea6-648bad7f5343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:09:29.316625 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:09:29.400659 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 758/1489] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:09:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:09:29.711123 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f981cb-bae0-4f09-831b-7477a704418f None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:09:29.712708 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-d8f981cb-bae0-4f09-831b-7477a704418f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:09:29.776155 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:09:29.911597 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f981cb-bae0-4f09-831b-7477a704418f None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:09:30.027815 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e059ce-3268-45c6-8e33-ab61687895e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:09:30.028559 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e059ce-3268-45c6-8e33-ab61687895e6 service neutron] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:09:30.028931 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e059ce-3268-45c6-8e33-ab61687895e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:09:30.029278 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e059ce-3268-45c6-8e33-ab61687895e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:09:30.031555 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-d7e059ce-3268-45c6-8e33-ab61687895e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:09:30.143807 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:09:30.183883 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e059ce-3268-45c6-8e33-ab61687895e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:09:30.185508 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e059ce-3268-45c6-8e33-ab61687895e6 service neutron] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:09:30.213632 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 732/1490] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:09:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:09:32.818515 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63211811-fa9d-4ae1-820e-5fa5823f6d87 None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:09:32.819097 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63211811-fa9d-4ae1-820e-5fa5823f6d87 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:09:32.819097 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63211811-fa9d-4ae1-820e-5fa5823f6d87 None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:09:32.820763 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 759/1491] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 6 20:09:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:09:32.827459 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a49d9b7-41b4-4fbf-b7e7-8746f399fcff None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:09:32.828347 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a49d9b7-41b4-4fbf-b7e7-8746f399fcff None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:09:32.828530 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a49d9b7-41b4-4fbf-b7e7-8746f399fcff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:09:32.924727 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-1a49d9b7-41b4-4fbf-b7e7-8746f399fcff None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:09:32.926349 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-1a49d9b7-41b4-4fbf-b7e7-8746f399fcff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:09:33.104315 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 733/1492] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 6 20:09:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:09:33.409662 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63211811-fa9d-4ae1-820e-5fa5823f6d87 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:09:33.412849 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-63211811-fa9d-4ae1-820e-5fa5823f6d87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:09:33.559495 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63211811-fa9d-4ae1-820e-5fa5823f6d87 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:09:33.680932 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7035cee6-caf6-4f1a-bbc1-3b73e8d88080 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:09:33.681564 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7035cee6-caf6-4f1a-bbc1-3b73e8d88080 service neutron] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:09:33.681972 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7035cee6-caf6-4f1a-bbc1-3b73e8d88080 service neutron] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:09:33.682318 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7035cee6-caf6-4f1a-bbc1-3b73e8d88080 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:09:33.684404 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-7035cee6-caf6-4f1a-bbc1-3b73e8d88080 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:09:33.697028 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:09:33.826481 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7035cee6-caf6-4f1a-bbc1-3b73e8d88080 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:09:33.828331 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7035cee6-caf6-4f1a-bbc1-3b73e8d88080 service neutron] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:09:33.850889 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 760/1493] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:09:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:09:43.102120 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=82 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:09:43.848268 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=50 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:10:59.448909 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b33424a-3224-435d-83f3-d0b938f7093c None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:10:59.449690 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b33424a-3224-435d-83f3-d0b938f7093c None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:10:59.449690 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b33424a-3224-435d-83f3-d0b938f7093c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:10:59.599895 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-8b33424a-3224-435d-83f3-d0b938f7093c None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:10:59.602245 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-8b33424a-3224-435d-83f3-d0b938f7093c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:10:59.748692 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 734/1494] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:10:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:10:59.758408 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aa8e9b-6378-4b22-93e1-a75ed5a8425c None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:10:59.759166 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aa8e9b-6378-4b22-93e1-a75ed5a8425c None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:10:59.759166 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aa8e9b-6378-4b22-93e1-a75ed5a8425c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:10:59.783102 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-71aa8e9b-6378-4b22-93e1-a75ed5a8425c None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:10:59.784612 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-71aa8e9b-6378-4b22-93e1-a75ed5a8425c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:10:59.794905 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 761/1495] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:10:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:10:59.800524 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b33424a-3224-435d-83f3-d0b938f7093c None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:10:59.812206 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b33424a-3224-435d-83f3-d0b938f7093c None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:10:59.817691 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8409f7ec-89f2-4134-9a1c-737e4e2b5953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:10:59.818347 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8409f7ec-89f2-4134-9a1c-737e4e2b5953 admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:10:59.818673 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8409f7ec-89f2-4134-9a1c-737e4e2b5953 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:10:59.818928 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8409f7ec-89f2-4134-9a1c-737e4e2b5953 admin admin] PATH_INFO: `/v3/domains` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:10:59.820135 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8409f7ec-89f2-4134-9a1c-737e4e2b5953 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:10:59.820437 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8409f7ec-89f2-4134-9a1c-737e4e2b5953 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:10:59.821578 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: WARNING py.warnings [None req-8409f7ec-89f2-4134-9a1c-737e4e2b5953 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 06 20:10:59.821578 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: warnings.warn(msg) Sep 06 20:10:59.821578 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: Sep 06 20:10:59.822516 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8409f7ec-89f2-4134-9a1c-737e4e2b5953 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:10:59.843022 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 735/1496] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 6 20:10:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:10:59.852734 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aa8e9b-6378-4b22-93e1-a75ed5a8425c None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:10:59.863917 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aa8e9b-6378-4b22-93e1-a75ed5a8425c None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:10:59.869289 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdb2b96-0a26-4001-8297-e51e2819b37e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:10:59.870048 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb2b96-0a26-4001-8297-e51e2819b37e admin admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:10:59.870429 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb2b96-0a26-4001-8297-e51e2819b37e admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:10:59.870787 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb2b96-0a26-4001-8297-e51e2819b37e admin admin] PATH_INFO: `/v3/projects` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:10:59.871498 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdb2b96-0a26-4001-8297-e51e2819b37e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:10:59.873521 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: WARNING py.warnings [None req-fbdb2b96-0a26-4001-8297-e51e2819b37e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 06 20:10:59.873521 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: warnings.warn(msg) Sep 06 20:10:59.873521 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: Sep 06 20:10:59.874819 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdb2b96-0a26-4001-8297-e51e2819b37e admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:10:59.926361 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 762/1497] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:10:59 2022] POST /identity/v3/projects => generated 379 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:10:59.931582 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8409f7ec-89f2-4134-9a1c-737e4e2b5953 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:10:59.946777 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8409f7ec-89f2-4134-9a1c-737e4e2b5953 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:11:00.037826 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:11:00.051938 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace47484-c91e-4951-9d48-f3908e14f581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:11:00.052978 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace47484-c91e-4951-9d48-f3908e14f581 admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:11:00.053633 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace47484-c91e-4951-9d48-f3908e14f581 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:11:00.054247 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace47484-c91e-4951-9d48-f3908e14f581 admin admin] PATH_INFO: `/v3/users` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:11:00.055531 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace47484-c91e-4951-9d48-f3908e14f581 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:11:00.057527 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: WARNING py.warnings [None req-ace47484-c91e-4951-9d48-f3908e14f581 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 06 20:11:00.057527 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: warnings.warn(msg) Sep 06 20:11:00.057527 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: Sep 06 20:11:00.059973 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace47484-c91e-4951-9d48-f3908e14f581 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:11:00.093348 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 736/1498] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:10:59 2022] POST /identity/v3/users => generated 290 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:11:00.097786 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdb2b96-0a26-4001-8297-e51e2819b37e admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:11:00.110792 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdb2b96-0a26-4001-8297-e51e2819b37e admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:11:00.116731 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc14c0a-053d-4c7f-b2e7-35e0501ffd54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:11:00.117473 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc14c0a-053d-4c7f-b2e7-35e0501ffd54 admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:11:00.117850 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc14c0a-053d-4c7f-b2e7-35e0501ffd54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:11:00.118171 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc14c0a-053d-4c7f-b2e7-35e0501ffd54 admin admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:11:00.119012 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc14c0a-053d-4c7f-b2e7-35e0501ffd54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:11:00.120174 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: WARNING py.warnings [None req-ddc14c0a-053d-4c7f-b2e7-35e0501ffd54 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 06 20:11:00.120174 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: warnings.warn(msg) Sep 06 20:11:00.120174 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: Sep 06 20:11:00.121430 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc14c0a-053d-4c7f-b2e7-35e0501ffd54 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:11:00.136854 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 763/1499] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:11:00 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 20:11:00.141647 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace47484-c91e-4951-9d48-f3908e14f581 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:11:00.156577 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace47484-c91e-4951-9d48-f3908e14f581 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:11:00.165195 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68015bc5-3140-4d02-9cde-41625f1d1d71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:11:00.166276 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68015bc5-3140-4d02-9cde-41625f1d1d71 admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:11:00.166747 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68015bc5-3140-4d02-9cde-41625f1d1d71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:11:00.167239 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68015bc5-3140-4d02-9cde-41625f1d1d71 admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:11:00.168440 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68015bc5-3140-4d02-9cde-41625f1d1d71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:11:00.170400 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: WARNING py.warnings [None req-68015bc5-3140-4d02-9cde-41625f1d1d71 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 06 20:11:00.170400 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: warnings.warn(msg) Sep 06 20:11:00.170400 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: Sep 06 20:11:00.172408 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68015bc5-3140-4d02-9cde-41625f1d1d71 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:11:00.200065 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 737/1500] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:11:00 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 20:11:00.205338 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc14c0a-053d-4c7f-b2e7-35e0501ffd54 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:11:00.218462 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc14c0a-053d-4c7f-b2e7-35e0501ffd54 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:11:00.224195 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e91d61a-760b-4f1c-b91b-9692f7ac0db5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:11:00.225000 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e91d61a-760b-4f1c-b91b-9692f7ac0db5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:11:00.225288 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e91d61a-760b-4f1c-b91b-9692f7ac0db5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:11:00.225805 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e91d61a-760b-4f1c-b91b-9692f7ac0db5 admin admin] PATH_INFO: `/v3/projects/2c27d201653945bc9ffb19ffe19439ce/users/ad94c215cd32430fb61c5e17467a4a89/roles/4eea39e24b06431aa357053fe7907a3d` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:11:00.245508 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e91d61a-760b-4f1c-b91b-9692f7ac0db5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c27d201653945bc9ffb19ffe19439ce, user_id=ad94c215cd32430fb61c5e17467a4a89, role_id=4eea39e24b06431aa357053fe7907a3d)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:11:00.246790 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: WARNING py.warnings [None req-0e91d61a-760b-4f1c-b91b-9692f7ac0db5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 06 20:11:00.246790 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: warnings.warn(msg) Sep 06 20:11:00.246790 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: Sep 06 20:11:00.249088 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e91d61a-760b-4f1c-b91b-9692f7ac0db5 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:11:00.265868 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 764/1501] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:11:00 2022] PUT /identity/v3/projects/2c27d201653945bc9ffb19ffe19439ce/users/ad94c215cd32430fb61c5e17467a4a89/roles/4eea39e24b06431aa357053fe7907a3d => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:11:00.279701 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a9bdc1-8340-497e-86b1-1c2081cb0385 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:11:00.280114 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a9bdc1-8340-497e-86b1-1c2081cb0385 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:11:00.280287 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a9bdc1-8340-497e-86b1-1c2081cb0385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:11:00.315927 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-40a9bdc1-8340-497e-86b1-1c2081cb0385 None None] MFA Rules not processed for user `ad94c215cd32430fb61c5e17467a4a89`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:11:00.317127 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-40a9bdc1-8340-497e-86b1-1c2081cb0385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:11:00.401105 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 738/1502] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:11:00 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:11:00.416762 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df02ea4-3614-45c2-abab-1b532f556f8f None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:11:00.417099 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df02ea4-3614-45c2-abab-1b532f556f8f None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:11:00.417422 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df02ea4-3614-45c2-abab-1b532f556f8f None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:11:00.419148 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 765/1503] 10.222.0.20 () {54 vars in 964 bytes} [Tue Sep 6 20:11:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:11:00.425799 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7776b32d-63e6-438a-a9ea-3882d90696b9 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:11:00.426257 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7776b32d-63e6-438a-a9ea-3882d90696b9 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:11:00.426647 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7776b32d-63e6-438a-a9ea-3882d90696b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:11:00.498203 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-7776b32d-63e6-438a-a9ea-3882d90696b9 None None] MFA Rules not processed for user `df1bca993bc142908cc7cf85eabd2b24`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:11:00.499489 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-7776b32d-63e6-438a-a9ea-3882d90696b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:11:00.599048 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 739/1504] 10.222.0.20 () {58 vars in 1061 bytes} [Tue Sep 6 20:11:00 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:11:00.610431 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df02ea4-3614-45c2-abab-1b532f556f8f None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:11:00.622798 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df02ea4-3614-45c2-abab-1b532f556f8f None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:11:00.626621 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d24503c-3145-4d5d-a60f-14f6cb84e8f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df1bca993bc142908cc7cf85eabd2b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:11:00.627247 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d24503c-3145-4d5d-a60f-14f6cb84e8f5 service cinder] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:11:00.627529 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d24503c-3145-4d5d-a60f-14f6cb84e8f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:11:00.627804 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d24503c-3145-4d5d-a60f-14f6cb84e8f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:11:00.638192 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d24503c-3145-4d5d-a60f-14f6cb84e8f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:11:00.639721 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d24503c-3145-4d5d-a60f-14f6cb84e8f5 service cinder] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:11:00.654216 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 766/1505] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:11:00 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 20:11:00.733391 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b3701c-702b-486c-aa38-d17f33abc405 None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:11:00.733732 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b3701c-702b-486c-aa38-d17f33abc405 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:11:00.734017 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b3701c-702b-486c-aa38-d17f33abc405 None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:11:00.735365 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 740/1506] 10.222.0.20 () {52 vars in 830 bytes} [Tue Sep 6 20:11:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 20:11:00.740242 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d24503c-3145-4d5d-a60f-14f6cb84e8f5 service cinder] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:11:00.817776 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d24503c-3145-4d5d-a60f-14f6cb84e8f5 service cinder] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:11:00.823772 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e90d2d2-6098-47fc-b740-3fee532408b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:11:00.824434 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e90d2d2-6098-47fc-b740-3fee532408b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:11:00.824719 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e90d2d2-6098-47fc-b740-3fee532408b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:11:00.824997 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e90d2d2-6098-47fc-b740-3fee532408b2 admin admin] PATH_INFO: `/v3/users/ad94c215cd32430fb61c5e17467a4a89` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:11:00.826253 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e90d2d2-6098-47fc-b740-3fee532408b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad94c215cd32430fb61c5e17467a4a89)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:11:00.827321 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: WARNING py.warnings [None req-6e90d2d2-6098-47fc-b740-3fee532408b2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 06 20:11:00.827321 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: warnings.warn(msg) Sep 06 20:11:00.827321 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: Sep 06 20:11:00.828561 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e90d2d2-6098-47fc-b740-3fee532408b2 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:11:00.925856 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-6e90d2d2-6098-47fc-b740-3fee532408b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad94c215cd32430fb61c5e17467a4a89'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:11:00.926872 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-6e90d2d2-6098-47fc-b740-3fee532408b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad94c215cd32430fb61c5e17467a4a89'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:11:00.960864 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-6e90d2d2-6098-47fc-b740-3fee532408b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad94c215cd32430fb61c5e17467a4a89'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:11:00.961743 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-6e90d2d2-6098-47fc-b740-3fee532408b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad94c215cd32430fb61c5e17467a4a89'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:11:00.979230 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-6e90d2d2-6098-47fc-b740-3fee532408b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad94c215cd32430fb61c5e17467a4a89'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:11:00.996224 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:11:01.016021 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 767/1507] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:11:00 2022] DELETE /identity/v3/users/ad94c215cd32430fb61c5e17467a4a89 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 20:11:01.022985 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ef4a54-f157-407c-9f38-ee8b1bd92590 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:11:01.023365 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ef4a54-f157-407c-9f38-ee8b1bd92590 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:11:01.023688 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ef4a54-f157-407c-9f38-ee8b1bd92590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:11:01.045768 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-67ef4a54-f157-407c-9f38-ee8b1bd92590 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:11:01.047321 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-67ef4a54-f157-407c-9f38-ee8b1bd92590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:11:01.060241 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:11:01.124513 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 741/1508] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:11:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:11:01.135874 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e90d2d2-6098-47fc-b740-3fee532408b2 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:11:01.137692 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-6e90d2d2-6098-47fc-b740-3fee532408b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:11:01.258965 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e90d2d2-6098-47fc-b740-3fee532408b2 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:11:01.263706 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c2e2c4-de0d-4db1-b0b3-99e25f8ca271 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:11:01.264479 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c2e2c4-de0d-4db1-b0b3-99e25f8ca271 service neutron] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:11:01.264890 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c2e2c4-de0d-4db1-b0b3-99e25f8ca271 service neutron] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:11:01.265285 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c2e2c4-de0d-4db1-b0b3-99e25f8ca271 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:11:01.277509 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c2e2c4-de0d-4db1-b0b3-99e25f8ca271 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:11:01.279259 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c2e2c4-de0d-4db1-b0b3-99e25f8ca271 service neutron] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:11:01.292191 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 768/1509] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:11:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:11:01.863401 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ef4a54-f157-407c-9f38-ee8b1bd92590 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:11:01.865446 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-67ef4a54-f157-407c-9f38-ee8b1bd92590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:11:01.881534 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ef4a54-f157-407c-9f38-ee8b1bd92590 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:11:01.888458 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c09a3b-e801-4856-a428-11b45ebb2e87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:11:01.889245 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c09a3b-e801-4856-a428-11b45ebb2e87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:11:01.890370 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c09a3b-e801-4856-a428-11b45ebb2e87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:11:01.890781 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c09a3b-e801-4856-a428-11b45ebb2e87 admin admin] PATH_INFO: `/v3/projects/2c27d201653945bc9ffb19ffe19439ce` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:11:01.893866 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c09a3b-e801-4856-a428-11b45ebb2e87 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c27d201653945bc9ffb19ffe19439ce)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:11:01.895706 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: WARNING py.warnings [None req-b8c09a3b-e801-4856-a428-11b45ebb2e87 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 06 20:11:01.895706 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: warnings.warn(msg) Sep 06 20:11:01.895706 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: Sep 06 20:11:01.898418 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c09a3b-e801-4856-a428-11b45ebb2e87 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:11:02.005833 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-b8c09a3b-e801-4856-a428-11b45ebb2e87 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c27d201653945bc9ffb19ffe19439ce', 'request_id': 'req-b8c09a3b-e801-4856-a428-11b45ebb2e87'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:11:02.022768 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-b8c09a3b-e801-4856-a428-11b45ebb2e87 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c27d201653945bc9ffb19ffe19439ce', 'request_id': 'req-b8c09a3b-e801-4856-a428-11b45ebb2e87'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:11:02.036004 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-b8c09a3b-e801-4856-a428-11b45ebb2e87 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c27d201653945bc9ffb19ffe19439ce', 'request_id': 'req-b8c09a3b-e801-4856-a428-11b45ebb2e87'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:11:02.051713 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-b8c09a3b-e801-4856-a428-11b45ebb2e87 admin admin] The token cache is being invalidate because project 2c27d201653945bc9ffb19ffe19439ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81771) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:11:02.052055 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-b8c09a3b-e801-4856-a428-11b45ebb2e87 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:11:02.053046 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-b8c09a3b-e801-4856-a428-11b45ebb2e87 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:11:02.056490 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 742/1510] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:11:01 2022] DELETE /identity/v3/projects/2c27d201653945bc9ffb19ffe19439ce => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 20:11:11.286558 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=38 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:11:12.048137 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=47,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:12:44.271774 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d31ec2-e2a6-4a1a-9f55-346f77351f8f None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:12:44.272445 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d31ec2-e2a6-4a1a-9f55-346f77351f8f None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:12:44.272445 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d31ec2-e2a6-4a1a-9f55-346f77351f8f None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:12:44.274839 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 769/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 6 20:12:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 20:12:44.280839 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ded967-e6f6-4a8e-96dc-c73f7c6097a7 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:12:44.281708 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ded967-e6f6-4a8e-96dc-c73f7c6097a7 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:12:44.281708 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ded967-e6f6-4a8e-96dc-c73f7c6097a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:12:44.311325 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-a8ded967-e6f6-4a8e-96dc-c73f7c6097a7 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:12:44.312643 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-a8ded967-e6f6-4a8e-96dc-c73f7c6097a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:12:44.391565 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 743/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 6 20:12:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:12:44.593762 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d31ec2-e2a6-4a1a-9f55-346f77351f8f None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:12:44.596191 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-42d31ec2-e2a6-4a1a-9f55-346f77351f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:12:44.725868 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d31ec2-e2a6-4a1a-9f55-346f77351f8f None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:12:44.733018 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238580e2-ee23-40cc-a260-9774396d40ac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5776cbf4f6714b2083c3ebdd6d98cc87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:12:44.733788 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238580e2-ee23-40cc-a260-9774396d40ac service nova] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:12:44.734122 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238580e2-ee23-40cc-a260-9774396d40ac service nova] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:12:44.734444 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238580e2-ee23-40cc-a260-9774396d40ac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:12:44.744730 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238580e2-ee23-40cc-a260-9774396d40ac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:12:44.746876 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238580e2-ee23-40cc-a260-9774396d40ac service nova] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:12:44.759984 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 770/1513] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:12:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 20:12:45.793803 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da4d383-b6a3-477b-a23b-b9b09322912b None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:12:45.794503 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da4d383-b6a3-477b-a23b-b9b09322912b None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:12:45.794738 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da4d383-b6a3-477b-a23b-b9b09322912b None None] PATH_INFO: `/v3` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:12:45.797602 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 744/1514] 10.222.0.98 () {54 vars in 925 bytes} [Tue Sep 6 20:12:45 2022] GET /identity/v3 => generated 255 bytes in 6 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:12:45.806809 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8dcb06-df75-46a1-a1c2-d210b0a9f4a9 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:12:45.807584 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8dcb06-df75-46a1-a1c2-d210b0a9f4a9 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:12:45.807978 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8dcb06-df75-46a1-a1c2-d210b0a9f4a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:12:45.900071 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-6b8dcb06-df75-46a1-a1c2-d210b0a9f4a9 None None] MFA Rules not processed for user `f57872de7bb74ea48d30e93e51830aba`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:12:45.901360 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-6b8dcb06-df75-46a1-a1c2-d210b0a9f4a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:12:46.079420 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90853097-6984-45cf-98c0-564062c33ec3 None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:12:46.080230 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90853097-6984-45cf-98c0-564062c33ec3 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:12:46.080823 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90853097-6984-45cf-98c0-564062c33ec3 None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:12:46.083726 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 745/1515] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 6 20:12:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:12:46.095329 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc1a4a9-cf97-4531-9b4f-35bde2d468fc None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:12:46.096053 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc1a4a9-cf97-4531-9b4f-35bde2d468fc None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:12:46.096466 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc1a4a9-cf97-4531-9b4f-35bde2d468fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:12:46.127191 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-4bc1a4a9-cf97-4531-9b4f-35bde2d468fc None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:12:46.128696 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-4bc1a4a9-cf97-4531-9b4f-35bde2d468fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:12:46.139633 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 746/1516] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 6 20:12:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:12:46.148734 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 771/1517] 10.222.0.98 () {58 vars in 1015 bytes} [Tue Sep 6 20:12:45 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 345 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:12:46.305029 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc1a4a9-cf97-4531-9b4f-35bde2d468fc None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:12:46.319182 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc1a4a9-cf97-4531-9b4f-35bde2d468fc None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:12:46.327346 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92a1e05-531d-4e12-9d79-b67a8934d99a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5776cbf4f6714b2083c3ebdd6d98cc87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:12:46.327992 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92a1e05-531d-4e12-9d79-b67a8934d99a service nova] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:12:46.328390 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92a1e05-531d-4e12-9d79-b67a8934d99a service nova] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:12:46.328757 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92a1e05-531d-4e12-9d79-b67a8934d99a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:12:46.339531 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92a1e05-531d-4e12-9d79-b67a8934d99a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:12:46.341254 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92a1e05-531d-4e12-9d79-b67a8934d99a service nova] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:12:46.352843 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 747/1518] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:12:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 20:12:55.009398 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1139fe7-5429-49b8-a859-fc95932bb063 None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:12:55.010049 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1139fe7-5429-49b8-a859-fc95932bb063 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:12:55.010511 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1139fe7-5429-49b8-a859-fc95932bb063 None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:12:55.013041 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 772/1519] 10.222.0.20 () {54 vars in 961 bytes} [Tue Sep 6 20:12:55 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:12:55.022532 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fd44fa-3426-4ce9-9fa7-d9e30b65af1a None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:12:55.023328 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fd44fa-3426-4ce9-9fa7-d9e30b65af1a None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:12:55.023654 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fd44fa-3426-4ce9-9fa7-d9e30b65af1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:12:55.056344 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-67fd44fa-3426-4ce9-9fa7-d9e30b65af1a None None] MFA Rules not processed for user `f57872de7bb74ea48d30e93e51830aba`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:12:55.058008 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-67fd44fa-3426-4ce9-9fa7-d9e30b65af1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:12:55.068300 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 748/1520] 10.222.0.20 () {58 vars in 1058 bytes} [Tue Sep 6 20:12:55 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:12:55.077563 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1139fe7-5429-49b8-a859-fc95932bb063 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:12:55.102521 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1139fe7-5429-49b8-a859-fc95932bb063 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:12:55.111430 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e378c72e-8868-45f0-ae3d-b09bd6a12e6a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f57872de7bb74ea48d30e93e51830aba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:12:55.112534 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e378c72e-8868-45f0-ae3d-b09bd6a12e6a service placement] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:12:55.113073 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e378c72e-8868-45f0-ae3d-b09bd6a12e6a service placement] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:12:55.113669 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e378c72e-8868-45f0-ae3d-b09bd6a12e6a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:12:55.131107 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e378c72e-8868-45f0-ae3d-b09bd6a12e6a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:12:55.134411 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e378c72e-8868-45f0-ae3d-b09bd6a12e6a service placement] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:12:55.154704 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 773/1521] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:12:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:12:58.182612 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ce552a-b485-4b92-89ba-f5006912cac8 None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:12:58.183168 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ce552a-b485-4b92-89ba-f5006912cac8 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:12:58.183168 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ce552a-b485-4b92-89ba-f5006912cac8 None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:12:58.184704 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 749/1522] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 6 20:12:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:12:58.192847 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47d9ecc-a619-4214-a432-b4302c8c9bac None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:12:58.193472 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47d9ecc-a619-4214-a432-b4302c8c9bac None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:12:58.193472 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47d9ecc-a619-4214-a432-b4302c8c9bac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:12:58.214931 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-b47d9ecc-a619-4214-a432-b4302c8c9bac None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:12:58.216300 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-b47d9ecc-a619-4214-a432-b4302c8c9bac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:12:58.226357 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 774/1523] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 6 20:12:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:12:58.395563 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ce552a-b485-4b92-89ba-f5006912cac8 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:12:58.412556 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ce552a-b485-4b92-89ba-f5006912cac8 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:12:58.418852 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15467e61-3760-42ea-904d-741a635ae957 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5776cbf4f6714b2083c3ebdd6d98cc87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:12:58.419384 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15467e61-3760-42ea-904d-741a635ae957 service nova] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:12:58.419587 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15467e61-3760-42ea-904d-741a635ae957 service nova] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:12:58.419957 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15467e61-3760-42ea-904d-741a635ae957 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:12:58.430664 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15467e61-3760-42ea-904d-741a635ae957 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:12:58.432331 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15467e61-3760-42ea-904d-741a635ae957 service nova] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:12:58.444372 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 750/1524] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:12:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:12:59.746901 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67216cb-cf1e-4b2b-bd69-b9874520119b None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:12:59.747529 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67216cb-cf1e-4b2b-bd69-b9874520119b None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:12:59.747870 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67216cb-cf1e-4b2b-bd69-b9874520119b None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:12:59.750600 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 775/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 6 20:12:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:12:59.758064 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd820b2-c072-4aa9-8e6c-aa3ffd995ee2 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:12:59.758701 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd820b2-c072-4aa9-8e6c-aa3ffd995ee2 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:12:59.758956 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd820b2-c072-4aa9-8e6c-aa3ffd995ee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:12:59.783523 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-7cd820b2-c072-4aa9-8e6c-aa3ffd995ee2 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:12:59.785038 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-7cd820b2-c072-4aa9-8e6c-aa3ffd995ee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:12:59.803901 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 751/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 6 20:12:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:12:59.967417 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67216cb-cf1e-4b2b-bd69-b9874520119b None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:12:59.980247 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:12:59.988592 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67216cb-cf1e-4b2b-bd69-b9874520119b None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:12:59.994583 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe65786-b595-472a-b123-daa7c5cf5e80 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5776cbf4f6714b2083c3ebdd6d98cc87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:12:59.995353 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe65786-b595-472a-b123-daa7c5cf5e80 service nova] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:12:59.995833 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe65786-b595-472a-b123-daa7c5cf5e80 service nova] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:12:59.996418 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe65786-b595-472a-b123-daa7c5cf5e80 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:13:00.007129 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe65786-b595-472a-b123-daa7c5cf5e80 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:13:00.009128 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe65786-b595-472a-b123-daa7c5cf5e80 service nova] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:13:00.021065 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 776/1527] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:12:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 20:13:04.420098 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babb27f4-8196-4e3e-bae9-988341395ddd None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:13:04.421114 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babb27f4-8196-4e3e-bae9-988341395ddd None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:13:04.421114 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babb27f4-8196-4e3e-bae9-988341395ddd None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:13:04.422561 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 752/1528] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 20:13:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:13:04.429354 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b118c3b-37e8-4aa4-a0f0-1cdd3b0f3941 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:13:04.430022 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b118c3b-37e8-4aa4-a0f0-1cdd3b0f3941 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:13:04.430022 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b118c3b-37e8-4aa4-a0f0-1cdd3b0f3941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:13:04.452189 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-4b118c3b-37e8-4aa4-a0f0-1cdd3b0f3941 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:13:04.453505 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-4b118c3b-37e8-4aa4-a0f0-1cdd3b0f3941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:13:04.462450 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 777/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 20:13:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:13:04.773708 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babb27f4-8196-4e3e-bae9-988341395ddd None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:13:04.775806 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-babb27f4-8196-4e3e-bae9-988341395ddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:13:04.889731 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babb27f4-8196-4e3e-bae9-988341395ddd None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:13:04.894317 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89f0348-16f4-41d9-a22b-effe74abefd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:13:04.894944 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89f0348-16f4-41d9-a22b-effe74abefd0 service neutron] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:13:04.895222 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89f0348-16f4-41d9-a22b-effe74abefd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:13:04.895492 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89f0348-16f4-41d9-a22b-effe74abefd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:13:04.915104 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89f0348-16f4-41d9-a22b-effe74abefd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:13:04.917465 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89f0348-16f4-41d9-a22b-effe74abefd0 service neutron] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:13:04.931068 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 753/1530] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:13:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 20:13:07.408492 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f2fe7b-db79-44e8-b4fc-13be9cd91dab None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:13:07.409379 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f2fe7b-db79-44e8-b4fc-13be9cd91dab None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:13:07.409379 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f2fe7b-db79-44e8-b4fc-13be9cd91dab None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:13:07.410571 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 778/1531] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 20:13:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 20:13:07.417977 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f23903-1c19-45a1-a272-434785527578 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:13:07.418635 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f23903-1c19-45a1-a272-434785527578 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:13:07.418902 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f23903-1c19-45a1-a272-434785527578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:13:07.453424 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-c2f23903-1c19-45a1-a272-434785527578 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:13:07.455417 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-c2f23903-1c19-45a1-a272-434785527578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:13:07.469835 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 754/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 20:13:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:13:07.770170 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f2fe7b-db79-44e8-b4fc-13be9cd91dab None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:13:07.782871 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f2fe7b-db79-44e8-b4fc-13be9cd91dab None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:13:07.786749 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01a1899-198e-4d28-822c-5011958f3eb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:13:07.787368 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01a1899-198e-4d28-822c-5011958f3eb3 service neutron] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:13:07.788327 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01a1899-198e-4d28-822c-5011958f3eb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:13:07.788327 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01a1899-198e-4d28-822c-5011958f3eb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:13:07.800009 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01a1899-198e-4d28-822c-5011958f3eb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:13:07.801983 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01a1899-198e-4d28-822c-5011958f3eb3 service neutron] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:13:07.814924 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 779/1533] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:13:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:13:11.891349 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87de333f-90d6-4553-bb4f-c83a96139509 None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:13:11.892101 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87de333f-90d6-4553-bb4f-c83a96139509 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:13:11.892101 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87de333f-90d6-4553-bb4f-c83a96139509 None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:13:11.893708 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 755/1534] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 20:13:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 20:13:11.900059 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16734bc-2d89-4f4d-8156-b1794aa779e7 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:13:11.900611 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16734bc-2d89-4f4d-8156-b1794aa779e7 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:13:11.900611 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16734bc-2d89-4f4d-8156-b1794aa779e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:13:11.924151 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-c16734bc-2d89-4f4d-8156-b1794aa779e7 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:13:11.925767 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-c16734bc-2d89-4f4d-8156-b1794aa779e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:13:11.937025 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 780/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 20:13:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:13:12.255020 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87de333f-90d6-4553-bb4f-c83a96139509 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:13:12.257000 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-87de333f-90d6-4553-bb4f-c83a96139509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:13:12.263401 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=78 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:13:12.333749 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87de333f-90d6-4553-bb4f-c83a96139509 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:13:12.337775 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63111d3-3be0-4ce3-a70a-6fe989838b95 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df1bca993bc142908cc7cf85eabd2b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:13:12.338409 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63111d3-3be0-4ce3-a70a-6fe989838b95 service cinder] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:13:12.338687 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63111d3-3be0-4ce3-a70a-6fe989838b95 service cinder] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:13:12.338964 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63111d3-3be0-4ce3-a70a-6fe989838b95 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:13:12.349773 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63111d3-3be0-4ce3-a70a-6fe989838b95 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:13:12.351242 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63111d3-3be0-4ce3-a70a-6fe989838b95 service cinder] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:13:12.362810 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 756/1536] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:13:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:13:14.761918 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e51706-0cc2-407a-a5da-58e8f81a7141 None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:13:14.762485 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e51706-0cc2-407a-a5da-58e8f81a7141 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:13:14.762852 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e51706-0cc2-407a-a5da-58e8f81a7141 None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:13:14.766751 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 781/1537] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 20:13:14 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:13:14.773173 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5473e1-c8d6-411a-b6c8-5035afdefa17 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:13:14.773859 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5473e1-c8d6-411a-b6c8-5035afdefa17 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:13:14.773859 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5473e1-c8d6-411a-b6c8-5035afdefa17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:13:14.797135 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-ba5473e1-c8d6-411a-b6c8-5035afdefa17 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:13:14.798657 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-ba5473e1-c8d6-411a-b6c8-5035afdefa17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:13:14.807521 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 757/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 20:13:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:13:15.170768 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e51706-0cc2-407a-a5da-58e8f81a7141 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:13:15.173734 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-95e51706-0cc2-407a-a5da-58e8f81a7141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:13:15.197111 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e51706-0cc2-407a-a5da-58e8f81a7141 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:13:15.202801 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8130f18e-02ab-4574-a539-f5922f4dc5ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df1bca993bc142908cc7cf85eabd2b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:13:15.203428 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8130f18e-02ab-4574-a539-f5922f4dc5ab service cinder] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:13:15.203712 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8130f18e-02ab-4574-a539-f5922f4dc5ab service cinder] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:13:15.204020 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8130f18e-02ab-4574-a539-f5922f4dc5ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:13:15.221815 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8130f18e-02ab-4574-a539-f5922f4dc5ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:13:15.224758 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8130f18e-02ab-4574-a539-f5922f4dc5ab service cinder] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:13:15.244023 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 782/1539] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:13:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:13:24.790261 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=28 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:13:25.235331 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=23 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:14:47.729280 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e03a8c-7da0-4300-bc30-d427cbdd65bb None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:47.730183 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e03a8c-7da0-4300-bc30-d427cbdd65bb None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:47.730183 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e03a8c-7da0-4300-bc30-d427cbdd65bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:47.771644 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-76e03a8c-7da0-4300-bc30-d427cbdd65bb None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:14:47.773884 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-76e03a8c-7da0-4300-bc30-d427cbdd65bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:14:47.797890 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 758/1540] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:14:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:14:47.831907 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082cb7e5-e081-4735-92b9-395312ad2309 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:47.833484 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082cb7e5-e081-4735-92b9-395312ad2309 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:47.834312 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082cb7e5-e081-4735-92b9-395312ad2309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:47.879460 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781d2464-9b83-4068-a5c0-5310b8247ceb None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:47.879967 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781d2464-9b83-4068-a5c0-5310b8247ceb None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:47.880466 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781d2464-9b83-4068-a5c0-5310b8247ceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:47.884070 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-082cb7e5-e081-4735-92b9-395312ad2309 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:14:47.886958 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-082cb7e5-e081-4735-92b9-395312ad2309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:14:47.923387 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 783/1541] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:14:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:14:47.925642 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c3cabe-44f2-47ef-a57b-8032fc83e556 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:47.926139 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c3cabe-44f2-47ef-a57b-8032fc83e556 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:47.926634 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c3cabe-44f2-47ef-a57b-8032fc83e556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:47.964308 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-781d2464-9b83-4068-a5c0-5310b8247ceb None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:14:47.966694 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-781d2464-9b83-4068-a5c0-5310b8247ceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:14:48.008085 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 759/1542] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:14:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:14:48.009552 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-a1c3cabe-44f2-47ef-a57b-8032fc83e556 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:14:48.010806 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781d2464-9b83-4068-a5c0-5310b8247ceb None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:14:48.012327 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-a1c3cabe-44f2-47ef-a57b-8032fc83e556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:14:48.043238 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781d2464-9b83-4068-a5c0-5310b8247ceb None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:14:48.048751 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 784/1543] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:14:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:14:48.053148 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf761a2-2621-4f11-bee9-7c70e166a11b None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:48.054657 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf761a2-2621-4f11-bee9-7c70e166a11b None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:48.057467 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf761a2-2621-4f11-bee9-7c70e166a11b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:48.066914 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cfc20c-5587-441f-883f-643720ca7f96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:14:48.068503 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cfc20c-5587-441f-883f-643720ca7f96 admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:48.069752 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cfc20c-5587-441f-883f-643720ca7f96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:48.070657 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cfc20c-5587-441f-883f-643720ca7f96 admin admin] PATH_INFO: `/v3/domains` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:48.072115 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cfc20c-5587-441f-883f-643720ca7f96 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:14:48.074215 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cfc20c-5587-441f-883f-643720ca7f96 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:14:48.077739 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cfc20c-5587-441f-883f-643720ca7f96 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:14:48.134484 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 760/1544] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 6 20:14:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:14:48.141728 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-adf761a2-2621-4f11-bee9-7c70e166a11b None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:14:48.142586 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-adf761a2-2621-4f11-bee9-7c70e166a11b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:14:48.142976 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1962e91-e7b1-4526-b260-289a7d153322 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:48.143827 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1962e91-e7b1-4526-b260-289a7d153322 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:48.144584 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1962e91-e7b1-4526-b260-289a7d153322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:48.183664 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 785/1545] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:14:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:14:48.186089 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e47268f-9108-4149-8017-4d9c55265b8c None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:48.186610 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e47268f-9108-4149-8017-4d9c55265b8c None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:48.187093 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e47268f-9108-4149-8017-4d9c55265b8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:48.214486 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-f1962e91-e7b1-4526-b260-289a7d153322 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:14:48.217478 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-f1962e91-e7b1-4526-b260-289a7d153322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:14:48.255822 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-9e47268f-9108-4149-8017-4d9c55265b8c None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:14:48.258029 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-9e47268f-9108-4149-8017-4d9c55265b8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:14:48.261151 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 761/1546] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:14:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:14:48.262118 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1962e91-e7b1-4526-b260-289a7d153322 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:14:48.286045 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1962e91-e7b1-4526-b260-289a7d153322 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:14:48.297025 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 786/1547] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:14:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:14:48.299371 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d76116-c99b-49a3-a066-552afd6b679d None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:48.299896 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d76116-c99b-49a3-a066-552afd6b679d None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:48.300344 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d76116-c99b-49a3-a066-552afd6b679d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:48.314437 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b43d2bf-bdc4-4c27-9f10-602e7435a58c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:14:48.315693 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b43d2bf-bdc4-4c27-9f10-602e7435a58c admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:48.316163 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b43d2bf-bdc4-4c27-9f10-602e7435a58c admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:48.316617 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b43d2bf-bdc4-4c27-9f10-602e7435a58c admin admin] PATH_INFO: `/v3/projects` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:48.317881 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b43d2bf-bdc4-4c27-9f10-602e7435a58c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:14:48.320225 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: WARNING py.warnings [None req-6b43d2bf-bdc4-4c27-9f10-602e7435a58c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 06 20:14:48.320225 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: warnings.warn(msg) Sep 06 20:14:48.320225 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: Sep 06 20:14:48.322981 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b43d2bf-bdc4-4c27-9f10-602e7435a58c admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:14:48.360247 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-b1d76116-c99b-49a3-a066-552afd6b679d None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:14:48.367021 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-b1d76116-c99b-49a3-a066-552afd6b679d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:14:48.396765 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 787/1548] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:14:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:14:48.401628 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bf5799-0ca1-4a5a-8434-526d1441b4a4 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:48.401628 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bf5799-0ca1-4a5a-8434-526d1441b4a4 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:48.401628 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bf5799-0ca1-4a5a-8434-526d1441b4a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:48.416985 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 762/1549] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:14:48 2022] POST /identity/v3/projects => generated 393 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:14:48.422513 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b43d2bf-bdc4-4c27-9f10-602e7435a58c admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:14:48.429545 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-48bf5799-0ca1-4a5a-8434-526d1441b4a4 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:14:48.431113 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-48bf5799-0ca1-4a5a-8434-526d1441b4a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:14:48.511835 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b43d2bf-bdc4-4c27-9f10-602e7435a58c admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:14:48.518897 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c623fb1-59e3-4b32-862e-0acef9cde48d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:14:48.519523 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c623fb1-59e3-4b32-862e-0acef9cde48d admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:48.519791 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c623fb1-59e3-4b32-862e-0acef9cde48d admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:48.520085 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c623fb1-59e3-4b32-862e-0acef9cde48d admin admin] PATH_INFO: `/v3/domains` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:48.520871 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c623fb1-59e3-4b32-862e-0acef9cde48d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:14:48.521013 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 788/1550] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:14:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:14:48.521134 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c623fb1-59e3-4b32-862e-0acef9cde48d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:14:48.522405 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bf5799-0ca1-4a5a-8434-526d1441b4a4 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:14:48.522965 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c623fb1-59e3-4b32-862e-0acef9cde48d admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:14:48.538747 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 763/1551] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 6 20:14:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:14:48.540026 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bf5799-0ca1-4a5a-8434-526d1441b4a4 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:14:48.540252 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4f1d00-6222-4e19-a0be-293b7d916a21 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:48.540437 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4f1d00-6222-4e19-a0be-293b7d916a21 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:48.540686 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4f1d00-6222-4e19-a0be-293b7d916a21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:48.547960 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba7201f-f02f-4248-bf25-0b776c048e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:14:48.549117 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba7201f-f02f-4248-bf25-0b776c048e3d admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:48.549606 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba7201f-f02f-4248-bf25-0b776c048e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:48.550129 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba7201f-f02f-4248-bf25-0b776c048e3d admin admin] PATH_INFO: `/v3/domains` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:48.551363 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba7201f-f02f-4248-bf25-0b776c048e3d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:14:48.551858 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba7201f-f02f-4248-bf25-0b776c048e3d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:14:48.553714 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: WARNING py.warnings [None req-9ba7201f-f02f-4248-bf25-0b776c048e3d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 06 20:14:48.553714 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: warnings.warn(msg) Sep 06 20:14:48.553714 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: Sep 06 20:14:48.555090 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba7201f-f02f-4248-bf25-0b776c048e3d admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:14:48.564964 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-ea4f1d00-6222-4e19-a0be-293b7d916a21 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:14:48.566336 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-ea4f1d00-6222-4e19-a0be-293b7d916a21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:14:48.584092 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 764/1552] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:14:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:14:48.584344 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 789/1553] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 6 20:14:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:14:48.584451 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4f1d00-6222-4e19-a0be-293b7d916a21 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:14:48.584555 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f3fc40-ea35-4b64-a4f3-44e638947746 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:48.584555 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f3fc40-ea35-4b64-a4f3-44e638947746 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:48.584555 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f3fc40-ea35-4b64-a4f3-44e638947746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:48.597397 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4f1d00-6222-4e19-a0be-293b7d916a21 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:14:48.603631 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39656c36-7b8a-47f8-963a-001be89b12c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:14:48.604799 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39656c36-7b8a-47f8-963a-001be89b12c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:48.605080 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39656c36-7b8a-47f8-963a-001be89b12c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:48.605437 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39656c36-7b8a-47f8-963a-001be89b12c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:48.606114 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39656c36-7b8a-47f8-963a-001be89b12c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:14:48.606526 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39656c36-7b8a-47f8-963a-001be89b12c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:14:48.608189 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39656c36-7b8a-47f8-963a-001be89b12c1 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:14:48.608400 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-25f3fc40-ea35-4b64-a4f3-44e638947746 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:14:48.609886 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-25f3fc40-ea35-4b64-a4f3-44e638947746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:14:48.622894 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 790/1554] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:14:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:14:48.623928 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f3fc40-ea35-4b64-a4f3-44e638947746 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:14:48.645023 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 765/1555] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 6 20:14:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:14:48.645342 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f3fc40-ea35-4b64-a4f3-44e638947746 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:14:48.645528 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afb274b-bbb1-4024-93c7-d9eeae6a8970 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:48.645528 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afb274b-bbb1-4024-93c7-d9eeae6a8970 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:48.645528 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afb274b-bbb1-4024-93c7-d9eeae6a8970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:48.654856 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d026e18-9226-4144-a607-2a2ecbdebe67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:14:48.655499 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d026e18-9226-4144-a607-2a2ecbdebe67 admin admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:48.655815 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d026e18-9226-4144-a607-2a2ecbdebe67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:48.656094 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d026e18-9226-4144-a607-2a2ecbdebe67 admin admin] PATH_INFO: `/v3/users` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:48.656765 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d026e18-9226-4144-a607-2a2ecbdebe67 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:14:48.657950 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: WARNING py.warnings [None req-5d026e18-9226-4144-a607-2a2ecbdebe67 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 06 20:14:48.657950 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: warnings.warn(msg) Sep 06 20:14:48.657950 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: Sep 06 20:14:48.659256 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d026e18-9226-4144-a607-2a2ecbdebe67 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:14:48.670434 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-9afb274b-bbb1-4024-93c7-d9eeae6a8970 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:14:48.672381 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-9afb274b-bbb1-4024-93c7-d9eeae6a8970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:14:48.683800 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 766/1556] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:14:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:14:48.685661 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31c0d53-93d1-4b67-8e86-a076d698dea0 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:48.685859 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31c0d53-93d1-4b67-8e86-a076d698dea0 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:48.686302 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31c0d53-93d1-4b67-8e86-a076d698dea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:48.698087 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 791/1557] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:14:48 2022] POST /identity/v3/users => generated 297 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:14:48.700496 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d026e18-9226-4144-a607-2a2ecbdebe67 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:14:48.716848 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d026e18-9226-4144-a607-2a2ecbdebe67 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:14:48.717837 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-e31c0d53-93d1-4b67-8e86-a076d698dea0 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:14:48.719817 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-e31c0d53-93d1-4b67-8e86-a076d698dea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:14:48.724859 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3661886f-b274-402b-bb73-3dbb9ec0789c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:14:48.725918 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3661886f-b274-402b-bb73-3dbb9ec0789c admin admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:48.726404 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3661886f-b274-402b-bb73-3dbb9ec0789c admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:48.726879 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3661886f-b274-402b-bb73-3dbb9ec0789c admin admin] PATH_INFO: `/v3/projects` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:48.727909 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3661886f-b274-402b-bb73-3dbb9ec0789c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:14:48.732134 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3661886f-b274-402b-bb73-3dbb9ec0789c admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:14:48.735081 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 767/1558] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:14:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:14:48.737266 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a584c3-2566-41a1-a87f-bd13dbb937db None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:48.737835 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a584c3-2566-41a1-a87f-bd13dbb937db None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:48.738312 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a584c3-2566-41a1-a87f-bd13dbb937db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:48.771713 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 792/1559] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:14:48 2022] POST /identity/v3/projects => generated 381 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:14:48.772679 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3661886f-b274-402b-bb73-3dbb9ec0789c admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:14:48.778935 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-d4a584c3-2566-41a1-a87f-bd13dbb937db None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:14:48.781107 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-d4a584c3-2566-41a1-a87f-bd13dbb937db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:14:48.856857 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3661886f-b274-402b-bb73-3dbb9ec0789c admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:14:48.862138 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e38ab88-3e16-4aa0-8bf4-c44d015c4a51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:14:48.862777 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e38ab88-3e16-4aa0-8bf4-c44d015c4a51 admin admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:48.863056 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e38ab88-3e16-4aa0-8bf4-c44d015c4a51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:48.863330 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e38ab88-3e16-4aa0-8bf4-c44d015c4a51 admin admin] PATH_INFO: `/v3/projects` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:48.863986 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e38ab88-3e16-4aa0-8bf4-c44d015c4a51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:14:48.865977 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e38ab88-3e16-4aa0-8bf4-c44d015c4a51 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:14:48.882652 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:14:48.893293 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 768/1560] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:14:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:14:48.896105 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a58c5f-6a79-4aa7-a492-2127fd20791e None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:48.896774 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a58c5f-6a79-4aa7-a492-2127fd20791e None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:48.897304 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a58c5f-6a79-4aa7-a492-2127fd20791e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:48.905735 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 793/1561] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:14:48 2022] POST /identity/v3/projects => generated 375 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:14:48.905735 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e38ab88-3e16-4aa0-8bf4-c44d015c4a51 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:14:48.922885 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:14:48.931525 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-b6a58c5f-6a79-4aa7-a492-2127fd20791e None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:14:48.934122 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-b6a58c5f-6a79-4aa7-a492-2127fd20791e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:14:48.997467 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e38ab88-3e16-4aa0-8bf4-c44d015c4a51 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:14:49.002972 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffab3978-bc80-4508-b781-02efacc09cee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:14:49.003617 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffab3978-bc80-4508-b781-02efacc09cee admin admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:49.003895 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffab3978-bc80-4508-b781-02efacc09cee admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:49.004173 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffab3978-bc80-4508-b781-02efacc09cee admin admin] PATH_INFO: `/v3/projects` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:49.004839 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffab3978-bc80-4508-b781-02efacc09cee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:14:49.006965 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffab3978-bc80-4508-b781-02efacc09cee admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:14:49.025067 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 769/1562] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:14:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:14:49.027882 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be5d3c4-cf64-46f6-9b99-71f2b883088f None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:49.028198 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be5d3c4-cf64-46f6-9b99-71f2b883088f None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:49.028530 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be5d3c4-cf64-46f6-9b99-71f2b883088f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:49.051495 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 794/1563] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:14:48 2022] POST /identity/v3/projects => generated 385 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:14:49.052801 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffab3978-bc80-4508-b781-02efacc09cee admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:14:49.057476 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-8be5d3c4-cf64-46f6-9b99-71f2b883088f None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:14:49.059378 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-8be5d3c4-cf64-46f6-9b99-71f2b883088f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:14:49.153806 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffab3978-bc80-4508-b781-02efacc09cee admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:14:49.158927 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 770/1564] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:14:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:14:49.160060 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be5d3c4-cf64-46f6-9b99-71f2b883088f None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:14:49.163299 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6a27e3-43ca-447e-9363-bfd177739e73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:14:49.164359 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6a27e3-43ca-447e-9363-bfd177739e73 admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:49.164864 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6a27e3-43ca-447e-9363-bfd177739e73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:49.165361 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6a27e3-43ca-447e-9363-bfd177739e73 admin admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:49.167319 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6a27e3-43ca-447e-9363-bfd177739e73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:14:49.169884 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6a27e3-43ca-447e-9363-bfd177739e73 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:14:49.171884 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be5d3c4-cf64-46f6-9b99-71f2b883088f None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:14:49.177900 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388c6ba3-3cd3-4c3e-b1bd-65ada9c5d201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:14:49.178597 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388c6ba3-3cd3-4c3e-b1bd-65ada9c5d201 admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:49.178919 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388c6ba3-3cd3-4c3e-b1bd-65ada9c5d201 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:49.179217 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388c6ba3-3cd3-4c3e-b1bd-65ada9c5d201 admin admin] PATH_INFO: `/v3/domains` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:49.179972 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388c6ba3-3cd3-4c3e-b1bd-65ada9c5d201 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:14:49.180300 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388c6ba3-3cd3-4c3e-b1bd-65ada9c5d201 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:14:49.182919 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388c6ba3-3cd3-4c3e-b1bd-65ada9c5d201 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:14:49.184072 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 795/1565] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:14:49 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:14:49.188037 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6a27e3-43ca-447e-9363-bfd177739e73 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:14:49.207031 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 771/1566] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 6 20:14:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:14:49.207031 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388c6ba3-3cd3-4c3e-b1bd-65ada9c5d201 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:14:49.207379 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6a27e3-43ca-447e-9363-bfd177739e73 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:14:49.216129 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36720204-b0dd-4240-b197-b3c80270fc8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:14:49.217182 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36720204-b0dd-4240-b197-b3c80270fc8b admin admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:49.217747 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36720204-b0dd-4240-b197-b3c80270fc8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:49.218346 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36720204-b0dd-4240-b197-b3c80270fc8b admin admin] PATH_INFO: `/v3/users` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:49.220919 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36720204-b0dd-4240-b197-b3c80270fc8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:14:49.221097 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388c6ba3-3cd3-4c3e-b1bd-65ada9c5d201 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:14:49.223957 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36720204-b0dd-4240-b197-b3c80270fc8b admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:14:49.226020 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6d1658-d66e-4129-8efa-21082fd0a9cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:14:49.226703 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6d1658-d66e-4129-8efa-21082fd0a9cc admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:49.227033 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6d1658-d66e-4129-8efa-21082fd0a9cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:49.227316 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6d1658-d66e-4129-8efa-21082fd0a9cc admin admin] PATH_INFO: `/v3/domains` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:49.228016 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6d1658-d66e-4129-8efa-21082fd0a9cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:14:49.228347 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6d1658-d66e-4129-8efa-21082fd0a9cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:14:49.230294 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6d1658-d66e-4129-8efa-21082fd0a9cc admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:14:49.246387 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 772/1567] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 6 20:14:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:14:49.247429 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6d1658-d66e-4129-8efa-21082fd0a9cc admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:14:49.259448 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6d1658-d66e-4129-8efa-21082fd0a9cc admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:14:49.265210 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6718d225-57be-49ab-9823-125e1871a85a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:14:49.266293 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6718d225-57be-49ab-9823-125e1871a85a admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:49.266774 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6718d225-57be-49ab-9823-125e1871a85a admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:49.267309 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6718d225-57be-49ab-9823-125e1871a85a admin admin] PATH_INFO: `/v3/users` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:49.268299 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6718d225-57be-49ab-9823-125e1871a85a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:14:49.270954 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6718d225-57be-49ab-9823-125e1871a85a admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:14:49.271518 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 796/1568] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:14:49 2022] POST /identity/v3/users => generated 291 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:14:49.273372 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36720204-b0dd-4240-b197-b3c80270fc8b admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:14:49.296841 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36720204-b0dd-4240-b197-b3c80270fc8b admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:14:49.300786 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 773/1569] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:14:49 2022] POST /identity/v3/users => generated 288 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:14:49.301716 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6718d225-57be-49ab-9823-125e1871a85a admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:14:49.308174 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7807c06-23b1-432d-9fac-212c459f2c12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:14:49.309097 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7807c06-23b1-432d-9fac-212c459f2c12 admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:49.309502 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7807c06-23b1-432d-9fac-212c459f2c12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:49.309930 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7807c06-23b1-432d-9fac-212c459f2c12 admin admin] PATH_INFO: `/v3/domains` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:49.310678 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7807c06-23b1-432d-9fac-212c459f2c12 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:14:49.311027 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7807c06-23b1-432d-9fac-212c459f2c12 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:14:49.313014 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7807c06-23b1-432d-9fac-212c459f2c12 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:14:49.314327 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6718d225-57be-49ab-9823-125e1871a85a admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:14:49.320555 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b15d105-1cd5-4c89-bc94-93b270785888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:14:49.321388 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b15d105-1cd5-4c89-bc94-93b270785888 admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:49.321703 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b15d105-1cd5-4c89-bc94-93b270785888 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:49.321986 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b15d105-1cd5-4c89-bc94-93b270785888 admin admin] PATH_INFO: `/v3/users` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:49.322717 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b15d105-1cd5-4c89-bc94-93b270785888 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:14:49.324859 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b15d105-1cd5-4c89-bc94-93b270785888 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:14:49.330277 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 797/1570] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 6 20:14:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:14:49.330960 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7807c06-23b1-432d-9fac-212c459f2c12 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:14:49.342512 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7807c06-23b1-432d-9fac-212c459f2c12 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:14:49.348374 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79556656-1131-4c24-a1e2-0c966d406a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:14:49.349145 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79556656-1131-4c24-a1e2-0c966d406a0b admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:49.349470 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79556656-1131-4c24-a1e2-0c966d406a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:49.349793 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79556656-1131-4c24-a1e2-0c966d406a0b admin admin] PATH_INFO: `/v3/domains` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:49.350570 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79556656-1131-4c24-a1e2-0c966d406a0b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:14:49.350905 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79556656-1131-4c24-a1e2-0c966d406a0b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:14:49.352952 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79556656-1131-4c24-a1e2-0c966d406a0b admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:14:49.367616 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 798/1571] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 6 20:14:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:14:49.372751 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 774/1572] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:14:49 2022] POST /identity/v3/users => generated 293 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:14:49.372945 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79556656-1131-4c24-a1e2-0c966d406a0b admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:14:49.373146 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b15d105-1cd5-4c89-bc94-93b270785888 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:14:49.386164 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79556656-1131-4c24-a1e2-0c966d406a0b admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:14:49.389788 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b15d105-1cd5-4c89-bc94-93b270785888 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:14:49.391494 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74db4a5-1f1d-4624-b0b4-59c2e4461990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:14:49.392241 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74db4a5-1f1d-4624-b0b4-59c2e4461990 admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:49.392531 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74db4a5-1f1d-4624-b0b4-59c2e4461990 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:49.392804 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74db4a5-1f1d-4624-b0b4-59c2e4461990 admin admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:49.393477 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74db4a5-1f1d-4624-b0b4-59c2e4461990 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:14:49.395193 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74db4a5-1f1d-4624-b0b4-59c2e4461990 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:14:49.397442 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7512005-b30c-427a-b6d7-a14310375548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:14:49.398546 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7512005-b30c-427a-b6d7-a14310375548 admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:49.399055 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7512005-b30c-427a-b6d7-a14310375548 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:49.399568 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7512005-b30c-427a-b6d7-a14310375548 admin admin] PATH_INFO: `/v3/projects` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:49.400887 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7512005-b30c-427a-b6d7-a14310375548 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:14:49.405948 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7512005-b30c-427a-b6d7-a14310375548 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:14:49.409077 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 799/1573] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:14:49 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:14:49.413746 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74db4a5-1f1d-4624-b0b4-59c2e4461990 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:14:49.425849 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74db4a5-1f1d-4624-b0b4-59c2e4461990 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:14:49.431014 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619cb02b-3305-45f7-859d-557406d8cba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:14:49.431655 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619cb02b-3305-45f7-859d-557406d8cba1 admin admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:49.432051 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619cb02b-3305-45f7-859d-557406d8cba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:49.432328 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619cb02b-3305-45f7-859d-557406d8cba1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:49.433117 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619cb02b-3305-45f7-859d-557406d8cba1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:14:49.435339 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619cb02b-3305-45f7-859d-557406d8cba1 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:14:49.460599 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:14:49.465719 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 775/1574] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:14:49 2022] POST /identity/v3/projects => generated 379 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:14:49.465719 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7512005-b30c-427a-b6d7-a14310375548 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:14:49.469750 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 800/1575] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:14:49 2022] POST /identity/v3/projects => generated 371 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:14:49.475536 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619cb02b-3305-45f7-859d-557406d8cba1 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:14:49.513950 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:14:49.571731 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7512005-b30c-427a-b6d7-a14310375548 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:14:49.573741 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619cb02b-3305-45f7-859d-557406d8cba1 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:14:49.581552 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b0e3d9-8169-4738-9482-698039015364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:14:49.582581 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b0e3d9-8169-4738-9482-698039015364 admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:49.583052 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b0e3d9-8169-4738-9482-698039015364 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:49.583507 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b0e3d9-8169-4738-9482-698039015364 admin admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:49.584640 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b0e3d9-8169-4738-9482-698039015364 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:14:49.584833 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506e1d7d-c186-478f-adc5-343d8b6cbe73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:14:49.585805 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506e1d7d-c186-478f-adc5-343d8b6cbe73 admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:49.586250 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506e1d7d-c186-478f-adc5-343d8b6cbe73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:49.586691 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506e1d7d-c186-478f-adc5-343d8b6cbe73 admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:49.587594 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b0e3d9-8169-4738-9482-698039015364 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:14:49.587815 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506e1d7d-c186-478f-adc5-343d8b6cbe73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:14:49.590977 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506e1d7d-c186-478f-adc5-343d8b6cbe73 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:14:49.610145 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 801/1576] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:14:49 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:14:49.610145 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b0e3d9-8169-4738-9482-698039015364 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:14:49.613950 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 776/1577] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:14:49 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:14:49.615468 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506e1d7d-c186-478f-adc5-343d8b6cbe73 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:14:49.622752 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b0e3d9-8169-4738-9482-698039015364 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:14:49.629438 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f3e5b8-be4a-46bb-81bf-ec08d78c7319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:14:49.630490 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f3e5b8-be4a-46bb-81bf-ec08d78c7319 admin admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:49.630959 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f3e5b8-be4a-46bb-81bf-ec08d78c7319 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:49.631400 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f3e5b8-be4a-46bb-81bf-ec08d78c7319 admin admin] PATH_INFO: `/v3/projects` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:49.631637 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506e1d7d-c186-478f-adc5-343d8b6cbe73 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:14:49.632539 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f3e5b8-be4a-46bb-81bf-ec08d78c7319 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:14:49.636701 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f3e5b8-be4a-46bb-81bf-ec08d78c7319 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:14:49.640430 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4dab0f-5466-466e-9ec8-e1ddb49e229f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:14:49.641323 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4dab0f-5466-466e-9ec8-e1ddb49e229f admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:49.641844 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4dab0f-5466-466e-9ec8-e1ddb49e229f admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:49.642308 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4dab0f-5466-466e-9ec8-e1ddb49e229f admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:49.643418 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4dab0f-5466-466e-9ec8-e1ddb49e229f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:14:49.646126 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4dab0f-5466-466e-9ec8-e1ddb49e229f admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:14:49.664099 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 777/1578] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:14:49 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:14:49.667322 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4dab0f-5466-466e-9ec8-e1ddb49e229f admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:14:49.683932 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4dab0f-5466-466e-9ec8-e1ddb49e229f admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:14:49.693103 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 802/1579] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:14:49 2022] POST /identity/v3/projects => generated 389 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:14:49.693103 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f3e5b8-be4a-46bb-81bf-ec08d78c7319 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:14:49.695208 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3572cfaf-caf3-4d95-83ff-3f82b0df1d68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:14:49.696386 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3572cfaf-caf3-4d95-83ff-3f82b0df1d68 admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:49.696886 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3572cfaf-caf3-4d95-83ff-3f82b0df1d68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:49.697921 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3572cfaf-caf3-4d95-83ff-3f82b0df1d68 admin admin] PATH_INFO: `/v3/projects` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:49.699183 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3572cfaf-caf3-4d95-83ff-3f82b0df1d68 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:14:49.703119 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3572cfaf-caf3-4d95-83ff-3f82b0df1d68 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:14:49.742269 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 778/1580] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:14:49 2022] POST /identity/v3/projects => generated 357 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:14:49.742269 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3572cfaf-caf3-4d95-83ff-3f82b0df1d68 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:14:49.801452 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f3e5b8-be4a-46bb-81bf-ec08d78c7319 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:14:49.838974 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3572cfaf-caf3-4d95-83ff-3f82b0df1d68 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:14:49.846982 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2826b2bf-e01a-4ad8-a7f5-ee4eb384c239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:14:49.848048 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2826b2bf-e01a-4ad8-a7f5-ee4eb384c239 admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:49.848550 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2826b2bf-e01a-4ad8-a7f5-ee4eb384c239 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:49.849208 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2826b2bf-e01a-4ad8-a7f5-ee4eb384c239 admin admin] PATH_INFO: `/v3/users` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:49.850472 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2826b2bf-e01a-4ad8-a7f5-ee4eb384c239 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:14:49.854296 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2826b2bf-e01a-4ad8-a7f5-ee4eb384c239 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:14:49.883183 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f62348-1dd1-4ecb-8ea1-f7c32fc9a596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:14:49.884114 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f62348-1dd1-4ecb-8ea1-f7c32fc9a596 admin admin] REQUEST_METHOD: `PUT` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:49.884550 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f62348-1dd1-4ecb-8ea1-f7c32fc9a596 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:49.885432 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f62348-1dd1-4ecb-8ea1-f7c32fc9a596 admin admin] PATH_INFO: `/v3/projects/b005ff023a9f46fda3ceef605996c385/users/ecde3e4bfd8b4f3490edd3832cea9e48/roles/4eea39e24b06431aa357053fe7907a3d` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:49.888124 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 779/1581] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:14:49 2022] POST /identity/v3/users => generated 290 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:14:49.889120 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2826b2bf-e01a-4ad8-a7f5-ee4eb384c239 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:14:49.902359 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2826b2bf-e01a-4ad8-a7f5-ee4eb384c239 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:14:49.908269 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1c1a5c-5a4e-46ae-8310-e410a50270b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:14:49.908982 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1c1a5c-5a4e-46ae-8310-e410a50270b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:49.909305 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1c1a5c-5a4e-46ae-8310-e410a50270b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:49.909504 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f62348-1dd1-4ecb-8ea1-f7c32fc9a596 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b005ff023a9f46fda3ceef605996c385, user_id=ecde3e4bfd8b4f3490edd3832cea9e48, role_id=4eea39e24b06431aa357053fe7907a3d)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:14:49.909725 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1c1a5c-5a4e-46ae-8310-e410a50270b8 admin admin] PATH_INFO: `/v3/users` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:49.910276 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1c1a5c-5a4e-46ae-8310-e410a50270b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:14:49.912342 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1c1a5c-5a4e-46ae-8310-e410a50270b8 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:14:49.912685 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f62348-1dd1-4ecb-8ea1-f7c32fc9a596 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:14:49.937213 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 803/1582] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:14:49 2022] PUT /identity/v3/projects/b005ff023a9f46fda3ceef605996c385/users/ecde3e4bfd8b4f3490edd3832cea9e48/roles/4eea39e24b06431aa357053fe7907a3d => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:14:49.938787 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f62348-1dd1-4ecb-8ea1-f7c32fc9a596 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:14:49.946124 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:14:49.946679 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 780/1583] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:14:49 2022] POST /identity/v3/users => generated 286 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:14:49.947556 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1c1a5c-5a4e-46ae-8310-e410a50270b8 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:14:49.962703 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1c1a5c-5a4e-46ae-8310-e410a50270b8 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:14:50.044658 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f62348-1dd1-4ecb-8ea1-f7c32fc9a596 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:14:50.050459 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f99d55d-ae7c-45b2-a934-61168f6feb4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:14:50.051544 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f99d55d-ae7c-45b2-a934-61168f6feb4e admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:50.052182 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f99d55d-ae7c-45b2-a934-61168f6feb4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:50.052576 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f99d55d-ae7c-45b2-a934-61168f6feb4e admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:50.052838 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8ba03d-9504-4578-9760-23a1a86b7fe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:14:50.053783 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f99d55d-ae7c-45b2-a934-61168f6feb4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:14:50.053985 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8ba03d-9504-4578-9760-23a1a86b7fe9 admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:50.054465 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8ba03d-9504-4578-9760-23a1a86b7fe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:50.055022 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8ba03d-9504-4578-9760-23a1a86b7fe9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:50.056055 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f99d55d-ae7c-45b2-a934-61168f6feb4e admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:14:50.056226 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8ba03d-9504-4578-9760-23a1a86b7fe9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:14:50.059121 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:14:50.059992 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8ba03d-9504-4578-9760-23a1a86b7fe9 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:14:50.084450 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 781/1584] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:14:49 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:14:50.084727 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 804/1585] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:14:49 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:14:50.084901 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f99d55d-ae7c-45b2-a934-61168f6feb4e admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:14:50.085038 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8ba03d-9504-4578-9760-23a1a86b7fe9 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:14:50.098834 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8ba03d-9504-4578-9760-23a1a86b7fe9 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:14:50.102596 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f99d55d-ae7c-45b2-a934-61168f6feb4e admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:14:50.104879 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62ac11e-a2f4-43fd-962b-82caef016e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:14:50.105562 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ac11e-a2f4-43fd-962b-82caef016e3d admin admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:50.105882 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ac11e-a2f4-43fd-962b-82caef016e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:50.106319 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ac11e-a2f4-43fd-962b-82caef016e3d admin admin] PATH_INFO: `/v3/users` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:50.106988 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62ac11e-a2f4-43fd-962b-82caef016e3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:14:50.109078 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62ac11e-a2f4-43fd-962b-82caef016e3d admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:14:50.109829 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08005c39-7e7d-42d4-ae79-c97b13f11633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:14:50.110828 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08005c39-7e7d-42d4-ae79-c97b13f11633 admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:50.111253 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08005c39-7e7d-42d4-ae79-c97b13f11633 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:50.111706 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08005c39-7e7d-42d4-ae79-c97b13f11633 admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:50.112636 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08005c39-7e7d-42d4-ae79-c97b13f11633 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:14:50.114428 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08005c39-7e7d-42d4-ae79-c97b13f11633 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:14:50.132829 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 782/1586] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:14:50 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:14:50.135500 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08005c39-7e7d-42d4-ae79-c97b13f11633 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:14:50.141297 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 805/1587] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:14:50 2022] POST /identity/v3/users => generated 295 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:14:50.142245 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62ac11e-a2f4-43fd-962b-82caef016e3d admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:14:50.153539 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62ac11e-a2f4-43fd-962b-82caef016e3d admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:14:50.154899 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08005c39-7e7d-42d4-ae79-c97b13f11633 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:14:50.159186 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e62d4e-8b8e-4024-848d-4cc4e77250ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:14:50.159829 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e62d4e-8b8e-4024-848d-4cc4e77250ee admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:50.160109 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e62d4e-8b8e-4024-848d-4cc4e77250ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:50.160494 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e62d4e-8b8e-4024-848d-4cc4e77250ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:50.161276 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e62d4e-8b8e-4024-848d-4cc4e77250ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:14:50.163194 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e62d4e-8b8e-4024-848d-4cc4e77250ee admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:14:50.163436 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cb8d0e-6e1e-4b2c-9cb4-207a0f482829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:14:50.164510 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cb8d0e-6e1e-4b2c-9cb4-207a0f482829 admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:50.165021 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cb8d0e-6e1e-4b2c-9cb4-207a0f482829 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:50.165526 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cb8d0e-6e1e-4b2c-9cb4-207a0f482829 admin admin] PATH_INFO: `/v3/users` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:50.167286 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cb8d0e-6e1e-4b2c-9cb4-207a0f482829 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:14:50.171244 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cb8d0e-6e1e-4b2c-9cb4-207a0f482829 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:14:50.177387 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 806/1588] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:14:50 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:14:50.178838 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eb52a0-19a7-4cea-9fd9-de7fd3997d5e None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:50.179410 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eb52a0-19a7-4cea-9fd9-de7fd3997d5e None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:50.179410 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eb52a0-19a7-4cea-9fd9-de7fd3997d5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:50.207818 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 783/1589] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:14:50 2022] POST /identity/v3/users => generated 279 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:14:50.208839 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cb8d0e-6e1e-4b2c-9cb4-207a0f482829 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:14:50.211440 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-c4eb52a0-19a7-4cea-9fd9-de7fd3997d5e None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:14:50.212973 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-c4eb52a0-19a7-4cea-9fd9-de7fd3997d5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:14:50.229290 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 807/1590] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:14:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:14:50.229290 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eb52a0-19a7-4cea-9fd9-de7fd3997d5e None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:14:50.239882 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cb8d0e-6e1e-4b2c-9cb4-207a0f482829 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:14:50.242570 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eb52a0-19a7-4cea-9fd9-de7fd3997d5e None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:14:50.246792 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ae4d2a-dd98-41a1-bee7-c352bdac83a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:14:50.247711 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ae4d2a-dd98-41a1-bee7-c352bdac83a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:50.248025 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ae4d2a-dd98-41a1-bee7-c352bdac83a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:50.248300 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ae4d2a-dd98-41a1-bee7-c352bdac83a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:50.248974 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ae4d2a-dd98-41a1-bee7-c352bdac83a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:14:50.249963 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9034b611-b50f-4799-b501-d82cc63f09b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:14:50.250662 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ae4d2a-dd98-41a1-bee7-c352bdac83a7 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:14:50.250867 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9034b611-b50f-4799-b501-d82cc63f09b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:50.251253 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9034b611-b50f-4799-b501-d82cc63f09b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:50.251741 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9034b611-b50f-4799-b501-d82cc63f09b7 admin admin] PATH_INFO: `/v3/projects/27f27bdf0ac14f62b682027b05b219ee/users/f49ef54a38614cb880577626e65b0e32/roles/4eea39e24b06431aa357053fe7907a3d` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:50.268155 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 784/1591] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:14:50 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:14:50.269488 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ae4d2a-dd98-41a1-bee7-c352bdac83a7 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:14:50.272641 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9034b611-b50f-4799-b501-d82cc63f09b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27f27bdf0ac14f62b682027b05b219ee, user_id=f49ef54a38614cb880577626e65b0e32, role_id=4eea39e24b06431aa357053fe7907a3d)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:14:50.275610 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9034b611-b50f-4799-b501-d82cc63f09b7 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:14:50.286827 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ae4d2a-dd98-41a1-bee7-c352bdac83a7 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:14:50.295331 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 808/1592] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:14:50 2022] PUT /identity/v3/projects/27f27bdf0ac14f62b682027b05b219ee/users/f49ef54a38614cb880577626e65b0e32/roles/4eea39e24b06431aa357053fe7907a3d => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:14:50.296298 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9034b611-b50f-4799-b501-d82cc63f09b7 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:14:50.381960 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a898895f-31ac-4257-8744-41a857d2de5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:14:50.382950 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a898895f-31ac-4257-8744-41a857d2de5f admin admin] REQUEST_METHOD: `PUT` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:50.383086 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9034b611-b50f-4799-b501-d82cc63f09b7 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:14:50.383443 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a898895f-31ac-4257-8744-41a857d2de5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:50.383928 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a898895f-31ac-4257-8744-41a857d2de5f admin admin] PATH_INFO: `/v3/projects/9492a3a2d2d74a1eafee466d4e060c39/users/4fe3ec5734d54682ab918a4dfcaa3880/roles/4eea39e24b06431aa357053fe7907a3d` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:50.389786 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cc5e9c-211e-441e-9dfe-4fc4725da1ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:14:50.390871 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cc5e9c-211e-441e-9dfe-4fc4725da1ae admin admin] REQUEST_METHOD: `PUT` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:50.391292 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cc5e9c-211e-441e-9dfe-4fc4725da1ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:50.391662 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cc5e9c-211e-441e-9dfe-4fc4725da1ae admin admin] PATH_INFO: `/v3/projects/4a8966ff4bda461aa40251ac9740d172/users/ad400ce73b644938aef19e2da731c4dc/roles/4eea39e24b06431aa357053fe7907a3d` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:50.411358 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cc5e9c-211e-441e-9dfe-4fc4725da1ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a8966ff4bda461aa40251ac9740d172, user_id=ad400ce73b644938aef19e2da731c4dc, role_id=4eea39e24b06431aa357053fe7907a3d)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:14:50.412920 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a898895f-31ac-4257-8744-41a857d2de5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9492a3a2d2d74a1eafee466d4e060c39, user_id=4fe3ec5734d54682ab918a4dfcaa3880, role_id=4eea39e24b06431aa357053fe7907a3d)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:14:50.414127 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: WARNING py.warnings [None req-a898895f-31ac-4257-8744-41a857d2de5f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 06 20:14:50.414127 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: warnings.warn(msg) Sep 06 20:14:50.414127 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: Sep 06 20:14:50.415233 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cc5e9c-211e-441e-9dfe-4fc4725da1ae admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:14:50.416844 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a898895f-31ac-4257-8744-41a857d2de5f admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:14:50.434748 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 809/1593] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:14:50 2022] PUT /identity/v3/projects/4a8966ff4bda461aa40251ac9740d172/users/ad400ce73b644938aef19e2da731c4dc/roles/4eea39e24b06431aa357053fe7907a3d => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:14:50.435040 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 785/1594] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:14:50 2022] PUT /identity/v3/projects/9492a3a2d2d74a1eafee466d4e060c39/users/4fe3ec5734d54682ab918a4dfcaa3880/roles/4eea39e24b06431aa357053fe7907a3d => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:14:50.436081 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a898895f-31ac-4257-8744-41a857d2de5f admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:14:50.436409 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cc5e9c-211e-441e-9dfe-4fc4725da1ae admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:14:50.477075 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:14:50.520579 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a898895f-31ac-4257-8744-41a857d2de5f admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:14:50.529573 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9364b8-9517-483d-9a9a-036e0ca4832b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:14:50.530599 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9364b8-9517-483d-9a9a-036e0ca4832b admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:50.531110 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9364b8-9517-483d-9a9a-036e0ca4832b admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:50.531622 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9364b8-9517-483d-9a9a-036e0ca4832b admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:50.533064 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9364b8-9517-483d-9a9a-036e0ca4832b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:14:50.536415 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9364b8-9517-483d-9a9a-036e0ca4832b admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:14:50.538566 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cc5e9c-211e-441e-9dfe-4fc4725da1ae admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:14:50.543999 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eec1c16-d6b5-461f-bf8e-8c0c2f305fdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:14:50.544641 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eec1c16-d6b5-461f-bf8e-8c0c2f305fdf admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:50.544924 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eec1c16-d6b5-461f-bf8e-8c0c2f305fdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:50.545202 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eec1c16-d6b5-461f-bf8e-8c0c2f305fdf admin admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:50.545912 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eec1c16-d6b5-461f-bf8e-8c0c2f305fdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:14:50.547602 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eec1c16-d6b5-461f-bf8e-8c0c2f305fdf admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:14:50.560501 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 786/1595] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:14:50 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:14:50.560501 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9364b8-9517-483d-9a9a-036e0ca4832b admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:14:50.564847 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 810/1596] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:14:50 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:14:50.565256 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eec1c16-d6b5-461f-bf8e-8c0c2f305fdf admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:14:50.571784 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9364b8-9517-483d-9a9a-036e0ca4832b admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:14:50.577384 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bb1dde-75e8-4b2f-898f-1ea4996d9122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:14:50.578046 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bb1dde-75e8-4b2f-898f-1ea4996d9122 admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:50.578325 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bb1dde-75e8-4b2f-898f-1ea4996d9122 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:50.578595 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bb1dde-75e8-4b2f-898f-1ea4996d9122 admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:50.579365 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bb1dde-75e8-4b2f-898f-1ea4996d9122 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:14:50.580939 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bb1dde-75e8-4b2f-898f-1ea4996d9122 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:14:50.593515 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 787/1597] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:14:50 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:14:50.594510 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bb1dde-75e8-4b2f-898f-1ea4996d9122 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:14:50.611727 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bb1dde-75e8-4b2f-898f-1ea4996d9122 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:14:50.620933 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccc888a-7fe4-42d5-b036-925bbde136eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:14:50.622113 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccc888a-7fe4-42d5-b036-925bbde136eb admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:50.622569 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccc888a-7fe4-42d5-b036-925bbde136eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:50.623017 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccc888a-7fe4-42d5-b036-925bbde136eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:50.624170 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccc888a-7fe4-42d5-b036-925bbde136eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:14:50.627475 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccc888a-7fe4-42d5-b036-925bbde136eb admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:14:50.633145 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eec1c16-d6b5-461f-bf8e-8c0c2f305fdf admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:14:50.638956 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504be953-a5a6-4802-9a5f-a8051bc45d9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:14:50.639601 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504be953-a5a6-4802-9a5f-a8051bc45d9b service neutron] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:50.640343 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504be953-a5a6-4802-9a5f-a8051bc45d9b service neutron] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:50.640719 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504be953-a5a6-4802-9a5f-a8051bc45d9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:50.654496 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 788/1598] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:14:50 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:14:50.654496 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28358633-fb01-4b96-b419-69333b3b6d97 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:50.654496 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28358633-fb01-4b96-b419-69333b3b6d97 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:50.654496 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28358633-fb01-4b96-b419-69333b3b6d97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:50.659448 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504be953-a5a6-4802-9a5f-a8051bc45d9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:14:50.661383 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504be953-a5a6-4802-9a5f-a8051bc45d9b service neutron] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:14:50.674678 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 811/1599] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:14:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:14:50.676398 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e87dec-3032-4a16-bae8-622af1c99842 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:50.676747 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e87dec-3032-4a16-bae8-622af1c99842 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:50.677014 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e87dec-3032-4a16-bae8-622af1c99842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:50.688195 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-28358633-fb01-4b96-b419-69333b3b6d97 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:14:50.690742 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-28358633-fb01-4b96-b419-69333b3b6d97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:14:50.701555 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 789/1600] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:14:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:14:50.703120 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-71e87dec-3032-4a16-bae8-622af1c99842 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:14:50.705258 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-71e87dec-3032-4a16-bae8-622af1c99842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:14:50.707072 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7409b2c3-d0a2-4567-b506-5aaf5ec002d9 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:50.707395 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7409b2c3-d0a2-4567-b506-5aaf5ec002d9 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:50.707567 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7409b2c3-d0a2-4567-b506-5aaf5ec002d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:50.724073 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 812/1601] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:14:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:14:50.724073 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e87dec-3032-4a16-bae8-622af1c99842 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:14:50.725658 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:14:50.741745 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e87dec-3032-4a16-bae8-622af1c99842 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:14:50.749905 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-7409b2c3-d0a2-4567-b506-5aaf5ec002d9 None None] MFA Rules not processed for user `4fe3ec5734d54682ab918a4dfcaa3880`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:14:50.751365 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2d8084-b284-424e-9eea-a95b1ac8f16b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:14:50.752523 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2d8084-b284-424e-9eea-a95b1ac8f16b admin admin] REQUEST_METHOD: `PUT` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:50.753000 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2d8084-b284-424e-9eea-a95b1ac8f16b admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:50.753511 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2d8084-b284-424e-9eea-a95b1ac8f16b admin admin] PATH_INFO: `/v3/projects/d3f88491a2ee4a03b15cb8c88a9aaf9b/users/48cca45629ef4e09aac4262616e2792f/roles/4eea39e24b06431aa357053fe7907a3d` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:50.754571 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-7409b2c3-d0a2-4567-b506-5aaf5ec002d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:14:50.776042 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2d8084-b284-424e-9eea-a95b1ac8f16b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3f88491a2ee4a03b15cb8c88a9aaf9b, user_id=48cca45629ef4e09aac4262616e2792f, role_id=4eea39e24b06431aa357053fe7907a3d)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:14:50.779408 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2d8084-b284-424e-9eea-a95b1ac8f16b admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:14:50.807310 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 813/1602] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:14:50 2022] PUT /identity/v3/projects/d3f88491a2ee4a03b15cb8c88a9aaf9b/users/48cca45629ef4e09aac4262616e2792f/roles/4eea39e24b06431aa357053fe7907a3d => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:14:50.807310 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2d8084-b284-424e-9eea-a95b1ac8f16b admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:14:50.883981 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 790/1603] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:14:50 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:14:50.883981 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7409b2c3-d0a2-4567-b506-5aaf5ec002d9 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:14:50.905119 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7409b2c3-d0a2-4567-b506-5aaf5ec002d9 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:14:50.920624 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2d8084-b284-424e-9eea-a95b1ac8f16b admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:14:50.931898 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fee2eb2-477f-4fa1-90af-89a50ea9e0cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:14:50.933422 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fee2eb2-477f-4fa1-90af-89a50ea9e0cb admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:50.934098 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fee2eb2-477f-4fa1-90af-89a50ea9e0cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:50.934428 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fee2eb2-477f-4fa1-90af-89a50ea9e0cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:50.943757 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fee2eb2-477f-4fa1-90af-89a50ea9e0cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:14:50.943757 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fee2eb2-477f-4fa1-90af-89a50ea9e0cb admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:14:50.974206 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 814/1604] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:14:50 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:14:50.981402 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fee2eb2-477f-4fa1-90af-89a50ea9e0cb admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:14:50.998211 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fee2eb2-477f-4fa1-90af-89a50ea9e0cb admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:14:51.006094 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59eaebf2-71b9-4723-ad1f-83a2d4ec59d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:14:51.007210 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eaebf2-71b9-4723-ad1f-83a2d4ec59d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:51.007719 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eaebf2-71b9-4723-ad1f-83a2d4ec59d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:51.007992 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eaebf2-71b9-4723-ad1f-83a2d4ec59d7 admin admin] PATH_INFO: `/v3/projects/0816f35db1824aa8b0b64a6459030c51/users/e4c283c5066a48b092d1c1c97a375409/roles/4eea39e24b06431aa357053fe7907a3d` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:51.016412 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dc6738-47a9-4df1-87a5-8f03967b7ebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:14:51.017048 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dc6738-47a9-4df1-87a5-8f03967b7ebd admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:51.017432 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dc6738-47a9-4df1-87a5-8f03967b7ebd admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:51.017742 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dc6738-47a9-4df1-87a5-8f03967b7ebd admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:51.018413 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dc6738-47a9-4df1-87a5-8f03967b7ebd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:14:51.020206 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dc6738-47a9-4df1-87a5-8f03967b7ebd admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:14:51.035307 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 791/1605] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:14:50 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:14:51.035929 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59eaebf2-71b9-4723-ad1f-83a2d4ec59d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0816f35db1824aa8b0b64a6459030c51, user_id=e4c283c5066a48b092d1c1c97a375409, role_id=4eea39e24b06431aa357053fe7907a3d)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:14:51.036283 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dc6738-47a9-4df1-87a5-8f03967b7ebd admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:14:51.040461 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-b7dc6738-47a9-4df1-87a5-8f03967b7ebd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:14:51.043800 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59eaebf2-71b9-4723-ad1f-83a2d4ec59d7 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:14:51.083842 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 815/1606] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:14:50 2022] PUT /identity/v3/projects/0816f35db1824aa8b0b64a6459030c51/users/e4c283c5066a48b092d1c1c97a375409/roles/4eea39e24b06431aa357053fe7907a3d => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:14:51.083842 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59eaebf2-71b9-4723-ad1f-83a2d4ec59d7 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:14:51.087135 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-59eaebf2-71b9-4723-ad1f-83a2d4ec59d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:14:51.100639 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:14:51.154197 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dc6738-47a9-4df1-87a5-8f03967b7ebd admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:14:51.219572 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59eaebf2-71b9-4723-ad1f-83a2d4ec59d7 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:14:51.227479 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb895450-284c-4ac1-8869-daa29e91cc8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:14:51.230907 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb895450-284c-4ac1-8869-daa29e91cc8e service neutron] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:51.230907 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb895450-284c-4ac1-8869-daa29e91cc8e service neutron] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:51.230907 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb895450-284c-4ac1-8869-daa29e91cc8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:51.239762 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:14:51.250896 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b910c2-7b7b-49f1-bed5-1f57d6d1d0ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:14:51.251735 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b910c2-7b7b-49f1-bed5-1f57d6d1d0ee service neutron] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:51.252022 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b910c2-7b7b-49f1-bed5-1f57d6d1d0ee service neutron] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:51.252301 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b910c2-7b7b-49f1-bed5-1f57d6d1d0ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:51.352608 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb895450-284c-4ac1-8869-daa29e91cc8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:14:51.355414 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb895450-284c-4ac1-8869-daa29e91cc8e service neutron] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:14:51.356715 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b910c2-7b7b-49f1-bed5-1f57d6d1d0ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:14:51.359315 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b910c2-7b7b-49f1-bed5-1f57d6d1d0ee service neutron] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:14:51.368683 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 816/1607] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:14:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:14:51.370991 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe500965-b8c4-402d-87bd-e91bf4e055b3 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:51.371342 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe500965-b8c4-402d-87bd-e91bf4e055b3 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:51.371607 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe500965-b8c4-402d-87bd-e91bf4e055b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:51.377790 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 792/1608] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:14:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:14:51.378555 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b910c2-7b7b-49f1-bed5-1f57d6d1d0ee service neutron] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:14:51.390372 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b910c2-7b7b-49f1-bed5-1f57d6d1d0ee service neutron] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:14:51.397535 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-fe500965-b8c4-402d-87bd-e91bf4e055b3 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:14:51.397823 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62250ac-b869-4897-bb4e-e9685499ae4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:14:51.398669 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62250ac-b869-4897-bb4e-e9685499ae4b admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:51.399187 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62250ac-b869-4897-bb4e-e9685499ae4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:51.399639 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62250ac-b869-4897-bb4e-e9685499ae4b admin admin] PATH_INFO: `/v3/projects` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:51.401504 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-fe500965-b8c4-402d-87bd-e91bf4e055b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:14:51.401701 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62250ac-b869-4897-bb4e-e9685499ae4b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:14:51.406012 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62250ac-b869-4897-bb4e-e9685499ae4b admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:14:51.417676 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 817/1609] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:14:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:14:51.417676 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe500965-b8c4-402d-87bd-e91bf4e055b3 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:14:51.434425 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe500965-b8c4-402d-87bd-e91bf4e055b3 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:14:51.443646 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9723623-eab4-4e02-9d9e-66b50711be73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:14:51.444635 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9723623-eab4-4e02-9d9e-66b50711be73 admin admin] REQUEST_METHOD: `PUT` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:51.444987 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9723623-eab4-4e02-9d9e-66b50711be73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:51.445494 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9723623-eab4-4e02-9d9e-66b50711be73 admin admin] PATH_INFO: `/v3/projects/d0bdb5e8899b4766b797688ad00c7ce0/users/7be957f2c15c4f6f949b3f284a30624d/roles/4eea39e24b06431aa357053fe7907a3d` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:51.454306 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 793/1610] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:14:51 2022] POST /identity/v3/projects => generated 381 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:14:51.455831 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62250ac-b869-4897-bb4e-e9685499ae4b admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:14:51.471600 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9723623-eab4-4e02-9d9e-66b50711be73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0bdb5e8899b4766b797688ad00c7ce0, user_id=7be957f2c15c4f6f949b3f284a30624d, role_id=4eea39e24b06431aa357053fe7907a3d)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:14:51.475582 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9723623-eab4-4e02-9d9e-66b50711be73 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:14:51.507593 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 818/1611] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:14:51 2022] PUT /identity/v3/projects/d0bdb5e8899b4766b797688ad00c7ce0/users/7be957f2c15c4f6f949b3f284a30624d/roles/4eea39e24b06431aa357053fe7907a3d => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:14:51.510700 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3027f250-ebc7-41bd-b414-d36fb3b8e144 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:51.511123 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3027f250-ebc7-41bd-b414-d36fb3b8e144 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:51.511799 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3027f250-ebc7-41bd-b414-d36fb3b8e144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:51.567286 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-3027f250-ebc7-41bd-b414-d36fb3b8e144 None None] MFA Rules not processed for user `e4c283c5066a48b092d1c1c97a375409`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:14:51.570504 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-3027f250-ebc7-41bd-b414-d36fb3b8e144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:14:51.592916 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62250ac-b869-4897-bb4e-e9685499ae4b admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:14:51.702350 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 819/1612] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:14:51 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:14:51.702350 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3027f250-ebc7-41bd-b414-d36fb3b8e144 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:14:51.727589 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b7f0e7-3708-4568-a1b7-571727008d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:14:51.727950 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:14:51.729535 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b7f0e7-3708-4568-a1b7-571727008d16 admin admin] REQUEST_METHOD: `PUT` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:51.730281 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b7f0e7-3708-4568-a1b7-571727008d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:51.730925 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b7f0e7-3708-4568-a1b7-571727008d16 admin admin] PATH_INFO: `/v3/projects/f4638584c5cc46c9b74ef60549cdd7ae/users/02c4c24e5f2849fd975dc354511510f2/roles/4eea39e24b06431aa357053fe7907a3d` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:51.761667 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b7f0e7-3708-4568-a1b7-571727008d16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4638584c5cc46c9b74ef60549cdd7ae, user_id=02c4c24e5f2849fd975dc354511510f2, role_id=4eea39e24b06431aa357053fe7907a3d)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:14:51.767158 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b7f0e7-3708-4568-a1b7-571727008d16 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:14:51.795402 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 794/1613] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:14:51 2022] PUT /identity/v3/projects/f4638584c5cc46c9b74ef60549cdd7ae/users/02c4c24e5f2849fd975dc354511510f2/roles/4eea39e24b06431aa357053fe7907a3d => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:14:51.796713 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b7f0e7-3708-4568-a1b7-571727008d16 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:14:51.820127 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:14:51.828003 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3027f250-ebc7-41bd-b414-d36fb3b8e144 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:14:51.951197 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9ffc02-4713-44de-819b-950fab245064 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:14:51.952798 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9ffc02-4713-44de-819b-950fab245064 service neutron] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:51.953335 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9ffc02-4713-44de-819b-950fab245064 service neutron] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:51.954000 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9ffc02-4713-44de-819b-950fab245064 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:51.965660 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b7f0e7-3708-4568-a1b7-571727008d16 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:14:51.974658 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431e6af2-3e33-43b7-abc7-ba7b9a381815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:14:51.975414 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431e6af2-3e33-43b7-abc7-ba7b9a381815 admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:51.975693 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431e6af2-3e33-43b7-abc7-ba7b9a381815 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:51.975971 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431e6af2-3e33-43b7-abc7-ba7b9a381815 admin admin] PATH_INFO: `/v3/users` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:51.976799 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431e6af2-3e33-43b7-abc7-ba7b9a381815 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:14:51.977383 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9ffc02-4713-44de-819b-950fab245064 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:14:51.980649 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9ffc02-4713-44de-819b-950fab245064 service neutron] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:14:51.980846 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431e6af2-3e33-43b7-abc7-ba7b9a381815 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:14:52.007123 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 820/1614] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:14:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:14:52.011966 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c032a8ca-90ef-4aeb-bbbe-9d2abce77cd3 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:52.013064 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c032a8ca-90ef-4aeb-bbbe-9d2abce77cd3 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:52.013600 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c032a8ca-90ef-4aeb-bbbe-9d2abce77cd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:52.020505 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 795/1615] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:14:51 2022] POST /identity/v3/users => generated 297 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:14:52.022101 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431e6af2-3e33-43b7-abc7-ba7b9a381815 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:14:52.058145 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-c032a8ca-90ef-4aeb-bbbe-9d2abce77cd3 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:14:52.060889 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-c032a8ca-90ef-4aeb-bbbe-9d2abce77cd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:14:52.062774 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431e6af2-3e33-43b7-abc7-ba7b9a381815 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:14:52.070221 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23cfccf-0fb1-46b2-a4e5-91407f885978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:14:52.071274 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23cfccf-0fb1-46b2-a4e5-91407f885978 admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:52.071873 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23cfccf-0fb1-46b2-a4e5-91407f885978 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:52.072631 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23cfccf-0fb1-46b2-a4e5-91407f885978 admin admin] PATH_INFO: `/v3/projects` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:52.073708 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23cfccf-0fb1-46b2-a4e5-91407f885978 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:14:52.077245 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23cfccf-0fb1-46b2-a4e5-91407f885978 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:14:52.081422 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 821/1616] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:14:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:14:52.081422 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c23553-d0b6-4ea6-afdd-a92dba84e7ba None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:52.081851 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c23553-d0b6-4ea6-afdd-a92dba84e7ba None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:52.082429 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c23553-d0b6-4ea6-afdd-a92dba84e7ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:52.131614 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 796/1617] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:14:52 2022] POST /identity/v3/projects => generated 373 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:14:52.133225 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23cfccf-0fb1-46b2-a4e5-91407f885978 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:14:52.152982 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-a5c23553-d0b6-4ea6-afdd-a92dba84e7ba None None] MFA Rules not processed for user `02c4c24e5f2849fd975dc354511510f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:14:52.155406 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-a5c23553-d0b6-4ea6-afdd-a92dba84e7ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:14:52.281913 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23cfccf-0fb1-46b2-a4e5-91407f885978 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:14:52.298427 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd33fe0f-4431-49c2-aaad-188f10db3100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:14:52.299239 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd33fe0f-4431-49c2-aaad-188f10db3100 admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:52.299523 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd33fe0f-4431-49c2-aaad-188f10db3100 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:52.299800 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd33fe0f-4431-49c2-aaad-188f10db3100 admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:52.300549 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd33fe0f-4431-49c2-aaad-188f10db3100 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:14:52.304492 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd33fe0f-4431-49c2-aaad-188f10db3100 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:14:52.337526 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 822/1618] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:14:52 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:14:52.337938 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 797/1619] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:14:52 2022] GET /identity/v3/roles => generated 1420 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:14:52.337938 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd33fe0f-4431-49c2-aaad-188f10db3100 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:14:52.338234 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c23553-d0b6-4ea6-afdd-a92dba84e7ba None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:14:52.383543 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c23553-d0b6-4ea6-afdd-a92dba84e7ba None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:14:52.404540 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e245f3-0514-4722-9e9c-9d33f753ac54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:14:52.405645 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e245f3-0514-4722-9e9c-9d33f753ac54 admin admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:52.406156 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e245f3-0514-4722-9e9c-9d33f753ac54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:52.406643 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e245f3-0514-4722-9e9c-9d33f753ac54 admin admin] PATH_INFO: `/v3/users` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:52.407886 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e245f3-0514-4722-9e9c-9d33f753ac54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:14:52.414188 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e245f3-0514-4722-9e9c-9d33f753ac54 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:14:52.437416 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd33fe0f-4431-49c2-aaad-188f10db3100 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:14:52.443042 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8214fcc-a344-4d6c-adba-5009eda40200 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:14:52.443693 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8214fcc-a344-4d6c-adba-5009eda40200 service neutron] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:52.444002 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8214fcc-a344-4d6c-adba-5009eda40200 service neutron] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:52.444282 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8214fcc-a344-4d6c-adba-5009eda40200 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:52.460570 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8214fcc-a344-4d6c-adba-5009eda40200 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:14:52.462996 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8214fcc-a344-4d6c-adba-5009eda40200 service neutron] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:14:52.476764 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 798/1620] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:14:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:14:52.477827 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8214fcc-a344-4d6c-adba-5009eda40200 service neutron] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:14:52.490971 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8214fcc-a344-4d6c-adba-5009eda40200 service neutron] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:14:52.496864 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a203c758-f70e-452a-9326-01e8e5902a65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:14:52.497692 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a203c758-f70e-452a-9326-01e8e5902a65 admin admin] REQUEST_METHOD: `PUT` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:52.497985 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a203c758-f70e-452a-9326-01e8e5902a65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:52.498332 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a203c758-f70e-452a-9326-01e8e5902a65 admin admin] PATH_INFO: `/v3/projects/572acb575fed41879328d05805ae264e/users/202c3c30109f47fc9236ae3741cad37c/roles/e0ab41d8ae22474688b5361bcc75e0f0` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:52.498523 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 823/1621] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:14:52 2022] POST /identity/v3/users => generated 293 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:14:52.501777 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e245f3-0514-4722-9e9c-9d33f753ac54 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:14:52.516821 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:14:52.524643 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a203c758-f70e-452a-9326-01e8e5902a65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=572acb575fed41879328d05805ae264e, user_id=202c3c30109f47fc9236ae3741cad37c, role_id=e0ab41d8ae22474688b5361bcc75e0f0)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:14:52.531180 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a203c758-f70e-452a-9326-01e8e5902a65 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:14:52.552366 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e245f3-0514-4722-9e9c-9d33f753ac54 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:14:52.565582 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 799/1622] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:14:52 2022] PUT /identity/v3/projects/572acb575fed41879328d05805ae264e/users/202c3c30109f47fc9236ae3741cad37c/roles/e0ab41d8ae22474688b5361bcc75e0f0 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:14:52.571863 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a203c758-f70e-452a-9326-01e8e5902a65 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:14:52.684542 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:14:52.688634 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a203c758-f70e-452a-9326-01e8e5902a65 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:14:52.696437 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2fe24c-75ad-47c1-873f-fa4d8174a576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:14:52.696437 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2fe24c-75ad-47c1-873f-fa4d8174a576 admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:52.696807 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2fe24c-75ad-47c1-873f-fa4d8174a576 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:52.697475 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2fe24c-75ad-47c1-873f-fa4d8174a576 admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:52.698219 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2fe24c-75ad-47c1-873f-fa4d8174a576 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:14:52.700095 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2fe24c-75ad-47c1-873f-fa4d8174a576 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:14:52.716783 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 800/1623] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:14:52 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:14:52.717939 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2fe24c-75ad-47c1-873f-fa4d8174a576 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:14:52.736586 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2fe24c-75ad-47c1-873f-fa4d8174a576 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:14:52.749852 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa88849-006c-4ece-81a2-2521695cca59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:14:52.751079 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa88849-006c-4ece-81a2-2521695cca59 admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:52.751557 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa88849-006c-4ece-81a2-2521695cca59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:52.752027 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa88849-006c-4ece-81a2-2521695cca59 admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:52.753537 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa88849-006c-4ece-81a2-2521695cca59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:14:52.757111 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa88849-006c-4ece-81a2-2521695cca59 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:14:52.783301 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd34484-ab25-4899-9bb0-2504f25f70be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:14:52.784388 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd34484-ab25-4899-9bb0-2504f25f70be admin admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:52.784885 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd34484-ab25-4899-9bb0-2504f25f70be admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:52.785349 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd34484-ab25-4899-9bb0-2504f25f70be admin admin] PATH_INFO: `/v3/projects` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:52.789256 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 801/1624] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:14:52 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:14:52.790368 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa88849-006c-4ece-81a2-2521695cca59 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:14:52.791337 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd34484-ab25-4899-9bb0-2504f25f70be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:14:52.800461 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd34484-ab25-4899-9bb0-2504f25f70be admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:14:52.805994 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa88849-006c-4ece-81a2-2521695cca59 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:14:52.817855 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ff158a-4118-4874-a419-1c0a1ba6e606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:14:52.818769 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ff158a-4118-4874-a419-1c0a1ba6e606 admin admin] REQUEST_METHOD: `PUT` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:52.819702 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ff158a-4118-4874-a419-1c0a1ba6e606 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:52.820069 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ff158a-4118-4874-a419-1c0a1ba6e606 admin admin] PATH_INFO: `/v3/projects/64f6e0fd681c4e42a3fcc256c1123eef/users/afe77006175f4df8a164eba0330dad7c/roles/e0ab41d8ae22474688b5361bcc75e0f0` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:52.863568 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ff158a-4118-4874-a419-1c0a1ba6e606 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64f6e0fd681c4e42a3fcc256c1123eef, user_id=afe77006175f4df8a164eba0330dad7c, role_id=e0ab41d8ae22474688b5361bcc75e0f0)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:14:52.877116 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ff158a-4118-4874-a419-1c0a1ba6e606 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:14:52.886508 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 824/1625] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:14:52 2022] POST /identity/v3/projects => generated 355 bytes in 387 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:14:52.887862 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd34484-ab25-4899-9bb0-2504f25f70be admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:14:52.905747 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 802/1626] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:14:52 2022] PUT /identity/v3/projects/64f6e0fd681c4e42a3fcc256c1123eef/users/afe77006175f4df8a164eba0330dad7c/roles/e0ab41d8ae22474688b5361bcc75e0f0 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:14:52.909683 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ff158a-4118-4874-a419-1c0a1ba6e606 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:14:53.098688 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd34484-ab25-4899-9bb0-2504f25f70be admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:14:53.121375 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e85a367-af73-436e-95bc-dbe172fc04d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:14:53.122121 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e85a367-af73-436e-95bc-dbe172fc04d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:53.122481 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e85a367-af73-436e-95bc-dbe172fc04d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:53.123755 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e85a367-af73-436e-95bc-dbe172fc04d7 admin admin] PATH_INFO: `/v3/projects/572acb575fed41879328d05805ae264e/users/202c3c30109f47fc9236ae3741cad37c/roles/e0ab41d8ae22474688b5361bcc75e0f0` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:53.128309 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e85a367-af73-436e-95bc-dbe172fc04d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=572acb575fed41879328d05805ae264e, user_id=202c3c30109f47fc9236ae3741cad37c, role_id=e0ab41d8ae22474688b5361bcc75e0f0)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:14:53.129904 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ff158a-4118-4874-a419-1c0a1ba6e606 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:14:53.138573 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e85a367-af73-436e-95bc-dbe172fc04d7 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:14:53.149084 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afec81e-d07f-4a9f-8299-ece66915299e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:14:53.150283 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afec81e-d07f-4a9f-8299-ece66915299e admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:53.150788 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afec81e-d07f-4a9f-8299-ece66915299e admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:53.151286 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afec81e-d07f-4a9f-8299-ece66915299e admin admin] PATH_INFO: `/v3/users` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:53.152771 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afec81e-d07f-4a9f-8299-ece66915299e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:14:53.158845 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 825/1627] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:14:52 2022] PUT /identity/v3/projects/572acb575fed41879328d05805ae264e/users/202c3c30109f47fc9236ae3741cad37c/roles/e0ab41d8ae22474688b5361bcc75e0f0 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:14:53.170448 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afec81e-d07f-4a9f-8299-ece66915299e admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:14:53.173183 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e85a367-af73-436e-95bc-dbe172fc04d7 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:14:53.243244 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 803/1628] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:14:52 2022] POST /identity/v3/users => generated 284 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:14:53.248510 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e92824-eb78-472c-8a1a-da9616096646 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:53.249045 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e92824-eb78-472c-8a1a-da9616096646 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:53.249497 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e92824-eb78-472c-8a1a-da9616096646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:53.352711 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-03e92824-eb78-472c-8a1a-da9616096646 None None] MFA Rules not processed for user `202c3c30109f47fc9236ae3741cad37c`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:14:53.355280 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-03e92824-eb78-472c-8a1a-da9616096646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:14:53.421275 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e85a367-af73-436e-95bc-dbe172fc04d7 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:14:53.436506 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c757fe1-fe2f-436b-8fba-59144c2c39b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:14:53.438543 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c757fe1-fe2f-436b-8fba-59144c2c39b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:53.439599 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c757fe1-fe2f-436b-8fba-59144c2c39b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:53.439903 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c757fe1-fe2f-436b-8fba-59144c2c39b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:53.442102 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c757fe1-fe2f-436b-8fba-59144c2c39b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:14:53.449110 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c757fe1-fe2f-436b-8fba-59144c2c39b0 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:14:53.465558 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:14:53.488330 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 826/1629] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:14:53 2022] GET /identity/v3/roles => generated 1420 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:14:53.488330 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c757fe1-fe2f-436b-8fba-59144c2c39b0 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:14:53.507358 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c757fe1-fe2f-436b-8fba-59144c2c39b0 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:14:53.518456 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f94f78-edd0-4915-a54a-dc7bca77860d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:14:53.521845 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f94f78-edd0-4915-a54a-dc7bca77860d admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:53.522942 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f94f78-edd0-4915-a54a-dc7bca77860d admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:53.523264 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f94f78-edd0-4915-a54a-dc7bca77860d admin admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:53.525644 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f94f78-edd0-4915-a54a-dc7bca77860d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:14:53.536091 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f94f78-edd0-4915-a54a-dc7bca77860d admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:14:53.560576 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 827/1630] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:14:53 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:14:53.562063 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f94f78-edd0-4915-a54a-dc7bca77860d admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:14:53.609222 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 804/1631] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:14:53 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 366 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:14:53.624998 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f94f78-edd0-4915-a54a-dc7bca77860d admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:14:53.633000 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e92824-eb78-472c-8a1a-da9616096646 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:14:53.634659 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cea9696-de49-403c-a3f4-508c7161ab58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:14:53.635830 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cea9696-de49-403c-a3f4-508c7161ab58 admin admin] REQUEST_METHOD: `PUT` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:53.636320 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cea9696-de49-403c-a3f4-508c7161ab58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:53.636799 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cea9696-de49-403c-a3f4-508c7161ab58 admin admin] PATH_INFO: `/v3/projects/64f6e0fd681c4e42a3fcc256c1123eef/users/afe77006175f4df8a164eba0330dad7c/roles/e0ab41d8ae22474688b5361bcc75e0f0` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:53.647143 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cea9696-de49-403c-a3f4-508c7161ab58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64f6e0fd681c4e42a3fcc256c1123eef, user_id=afe77006175f4df8a164eba0330dad7c, role_id=e0ab41d8ae22474688b5361bcc75e0f0)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:14:53.647143 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cea9696-de49-403c-a3f4-508c7161ab58 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:14:53.671879 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 828/1632] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:14:53 2022] PUT /identity/v3/projects/64f6e0fd681c4e42a3fcc256c1123eef/users/afe77006175f4df8a164eba0330dad7c/roles/e0ab41d8ae22474688b5361bcc75e0f0 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:14:53.675509 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cea9696-de49-403c-a3f4-508c7161ab58 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:14:53.690392 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e92824-eb78-472c-8a1a-da9616096646 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:14:53.781154 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:14:53.861920 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca2d80b-96b3-44c2-865e-1e0f01f84036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:14:53.862681 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca2d80b-96b3-44c2-865e-1e0f01f84036 admin admin] REQUEST_METHOD: `PUT` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:53.863079 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca2d80b-96b3-44c2-865e-1e0f01f84036 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:53.863375 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca2d80b-96b3-44c2-865e-1e0f01f84036 admin admin] PATH_INFO: `/v3/projects/54061e8ce554448989fe72a89b243f9b/users/bd3e397544154877ab9cd65375189507/roles/e0ab41d8ae22474688b5361bcc75e0f0` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:53.869304 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cea9696-de49-403c-a3f4-508c7161ab58 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:14:53.887542 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca2d80b-96b3-44c2-865e-1e0f01f84036 admin admin] RBAC: Authorizing `identity:create_grant(project_id=54061e8ce554448989fe72a89b243f9b, user_id=bd3e397544154877ab9cd65375189507, role_id=e0ab41d8ae22474688b5361bcc75e0f0)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:14:53.891039 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca2d80b-96b3-44c2-865e-1e0f01f84036 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:14:53.892517 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d5d3ad-d2ed-4097-8ca2-53407b1eab97 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df1bca993bc142908cc7cf85eabd2b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:14:53.893753 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d5d3ad-d2ed-4097-8ca2-53407b1eab97 service cinder] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:53.894348 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d5d3ad-d2ed-4097-8ca2-53407b1eab97 service cinder] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:53.894935 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d5d3ad-d2ed-4097-8ca2-53407b1eab97 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:53.922625 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 805/1633] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:14:53 2022] PUT /identity/v3/projects/54061e8ce554448989fe72a89b243f9b/users/bd3e397544154877ab9cd65375189507/roles/e0ab41d8ae22474688b5361bcc75e0f0 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:14:53.924108 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d5b2e6-f27a-4ee3-8202-d0adc9b78cf6 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:53.924391 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d5b2e6-f27a-4ee3-8202-d0adc9b78cf6 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:53.924668 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d5b2e6-f27a-4ee3-8202-d0adc9b78cf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:53.989403 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-33d5b2e6-f27a-4ee3-8202-d0adc9b78cf6 None None] MFA Rules not processed for user `afe77006175f4df8a164eba0330dad7c`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:14:53.990992 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-33d5b2e6-f27a-4ee3-8202-d0adc9b78cf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:14:54.166039 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d5d3ad-d2ed-4097-8ca2-53407b1eab97 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:14:54.184427 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d5d3ad-d2ed-4097-8ca2-53407b1eab97 service cinder] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:14:54.228867 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 806/1634] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:14:53 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:14:54.265311 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d5b2e6-f27a-4ee3-8202-d0adc9b78cf6 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:14:54.265526 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 829/1635] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:14:53 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 592 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:14:54.274274 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d5d3ad-d2ed-4097-8ca2-53407b1eab97 service cinder] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:14:54.303944 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d5d3ad-d2ed-4097-8ca2-53407b1eab97 service cinder] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:14:54.399418 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:14:54.474386 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d5b2e6-f27a-4ee3-8202-d0adc9b78cf6 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:14:54.481532 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a4e54e-cafc-4a3b-a741-ce0348f230b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:14:54.482869 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a4e54e-cafc-4a3b-a741-ce0348f230b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:54.484502 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a4e54e-cafc-4a3b-a741-ce0348f230b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:54.485717 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a4e54e-cafc-4a3b-a741-ce0348f230b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:54.486503 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a4e54e-cafc-4a3b-a741-ce0348f230b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:14:54.487347 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef98e295-eed6-4d73-89aa-1fc05c0c690a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df1bca993bc142908cc7cf85eabd2b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:14:54.489198 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a4e54e-cafc-4a3b-a741-ce0348f230b3 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:14:54.490065 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef98e295-eed6-4d73-89aa-1fc05c0c690a service cinder] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:54.490453 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef98e295-eed6-4d73-89aa-1fc05c0c690a service cinder] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:54.490883 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef98e295-eed6-4d73-89aa-1fc05c0c690a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:54.520109 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef98e295-eed6-4d73-89aa-1fc05c0c690a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:14:54.523104 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 807/1636] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:14:54 2022] GET /identity/v3/roles => generated 1420 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:14:54.524644 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a4e54e-cafc-4a3b-a741-ce0348f230b3 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:14:54.525540 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef98e295-eed6-4d73-89aa-1fc05c0c690a service cinder] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:14:54.549252 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a4e54e-cafc-4a3b-a741-ce0348f230b3 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:14:54.557316 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 830/1637] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:14:54 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:14:54.559192 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef98e295-eed6-4d73-89aa-1fc05c0c690a service cinder] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:14:54.574914 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef8f904-b388-4007-b7a9-b3fa00849be8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df1bca993bc142908cc7cf85eabd2b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:14:54.584447 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef8f904-b388-4007-b7a9-b3fa00849be8 service cinder] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:54.586531 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef8f904-b388-4007-b7a9-b3fa00849be8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:54.587488 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef8f904-b388-4007-b7a9-b3fa00849be8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:54.625949 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef98e295-eed6-4d73-89aa-1fc05c0c690a service cinder] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:14:54.647726 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec6aa39-3f8e-415a-8ec0-7f5328a547fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:14:54.662456 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec6aa39-3f8e-415a-8ec0-7f5328a547fa admin admin] REQUEST_METHOD: `PUT` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:54.663157 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec6aa39-3f8e-415a-8ec0-7f5328a547fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:54.663486 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec6aa39-3f8e-415a-8ec0-7f5328a547fa admin admin] PATH_INFO: `/v3/projects/54061e8ce554448989fe72a89b243f9b/users/bd3e397544154877ab9cd65375189507/roles/e0ab41d8ae22474688b5361bcc75e0f0` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:54.680698 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec6aa39-3f8e-415a-8ec0-7f5328a547fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=54061e8ce554448989fe72a89b243f9b, user_id=bd3e397544154877ab9cd65375189507, role_id=e0ab41d8ae22474688b5361bcc75e0f0)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:14:54.686605 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec6aa39-3f8e-415a-8ec0-7f5328a547fa admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:14:54.736626 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 831/1638] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:14:54 2022] PUT /identity/v3/projects/54061e8ce554448989fe72a89b243f9b/users/bd3e397544154877ab9cd65375189507/roles/e0ab41d8ae22474688b5361bcc75e0f0 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:14:54.748991 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59201020-abaa-41f7-908c-658f291c7a70 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:54.749480 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59201020-abaa-41f7-908c-658f291c7a70 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:54.750000 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59201020-abaa-41f7-908c-658f291c7a70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:54.806061 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef8f904-b388-4007-b7a9-b3fa00849be8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:14:54.809085 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef8f904-b388-4007-b7a9-b3fa00849be8 service cinder] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:14:54.844384 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 808/1639] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:14:54 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:14:54.848073 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-59201020-abaa-41f7-908c-658f291c7a70 None None] MFA Rules not processed for user `bd3e397544154877ab9cd65375189507`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:14:54.855401 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-59201020-abaa-41f7-908c-658f291c7a70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:14:54.964064 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:14:55.084664 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 832/1640] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:14:54 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:14:55.108810 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef8f904-b388-4007-b7a9-b3fa00849be8 service cinder] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:14:55.161460 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef8f904-b388-4007-b7a9-b3fa00849be8 service cinder] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:14:55.335204 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f3cd13-9eac-47e7-b701-1dc536a895ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df1bca993bc142908cc7cf85eabd2b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:14:55.337746 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f3cd13-9eac-47e7-b701-1dc536a895ab service cinder] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:55.338781 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f3cd13-9eac-47e7-b701-1dc536a895ab service cinder] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:55.339792 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f3cd13-9eac-47e7-b701-1dc536a895ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:55.385131 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f3cd13-9eac-47e7-b701-1dc536a895ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:14:55.404274 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f3cd13-9eac-47e7-b701-1dc536a895ab service cinder] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:14:55.459166 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 809/1641] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:14:55 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:14:55.657034 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59201020-abaa-41f7-908c-658f291c7a70 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:14:55.684587 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59201020-abaa-41f7-908c-658f291c7a70 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:14:55.702254 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b25d58-d990-486e-8340-9f7e57a4c4c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df1bca993bc142908cc7cf85eabd2b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:14:55.703366 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b25d58-d990-486e-8340-9f7e57a4c4c2 service cinder] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:55.703901 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b25d58-d990-486e-8340-9f7e57a4c4c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:55.704426 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b25d58-d990-486e-8340-9f7e57a4c4c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:55.842036 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b25d58-d990-486e-8340-9f7e57a4c4c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:14:55.845845 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b25d58-d990-486e-8340-9f7e57a4c4c2 service cinder] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:14:55.859727 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 833/1642] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:14:55 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:14:56.190971 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f3cd13-9eac-47e7-b701-1dc536a895ab service cinder] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:14:56.287934 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f3cd13-9eac-47e7-b701-1dc536a895ab service cinder] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:14:56.295492 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca3b331-b50c-460e-897b-2beaf1586256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:14:56.296619 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca3b331-b50c-460e-897b-2beaf1586256 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:56.298361 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca3b331-b50c-460e-897b-2beaf1586256 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:56.298950 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca3b331-b50c-460e-897b-2beaf1586256 admin admin] PATH_INFO: `/v3/users/e4c283c5066a48b092d1c1c97a375409` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:56.301410 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca3b331-b50c-460e-897b-2beaf1586256 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4c283c5066a48b092d1c1c97a375409)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:14:56.302924 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: WARNING py.warnings [None req-fca3b331-b50c-460e-897b-2beaf1586256 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 06 20:14:56.302924 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: warnings.warn(msg) Sep 06 20:14:56.302924 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: Sep 06 20:14:56.305040 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca3b331-b50c-460e-897b-2beaf1586256 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:14:56.327487 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:14:56.420564 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-fca3b331-b50c-460e-897b-2beaf1586256 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4c283c5066a48b092d1c1c97a375409'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:14:56.434316 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-fca3b331-b50c-460e-897b-2beaf1586256 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4c283c5066a48b092d1c1c97a375409'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:14:56.435508 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-fca3b331-b50c-460e-897b-2beaf1586256 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4c283c5066a48b092d1c1c97a375409'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:14:56.437345 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-fca3b331-b50c-460e-897b-2beaf1586256 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4c283c5066a48b092d1c1c97a375409'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:14:56.469296 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-fca3b331-b50c-460e-897b-2beaf1586256 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4c283c5066a48b092d1c1c97a375409'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:14:56.520070 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 810/1643] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:14:56 2022] DELETE /identity/v3/users/e4c283c5066a48b092d1c1c97a375409 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:14:56.525573 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af2d4af-65ff-4c36-8028-6d6bde31f287 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:56.525883 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af2d4af-65ff-4c36-8028-6d6bde31f287 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:56.526162 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af2d4af-65ff-4c36-8028-6d6bde31f287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:56.574596 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-7af2d4af-65ff-4c36-8028-6d6bde31f287 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:14:56.575963 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-7af2d4af-65ff-4c36-8028-6d6bde31f287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:14:56.817322 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 834/1644] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:14:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:14:56.893247 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca3b331-b50c-460e-897b-2beaf1586256 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:14:56.896011 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-fca3b331-b50c-460e-897b-2beaf1586256 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:14:57.146950 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca3b331-b50c-460e-897b-2beaf1586256 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:14:57.160470 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b26f7c-8376-4593-b36f-b197603d73f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:14:57.163341 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b26f7c-8376-4593-b36f-b197603d73f1 service neutron] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:14:57.163774 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b26f7c-8376-4593-b36f-b197603d73f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:14:57.170632 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b26f7c-8376-4593-b36f-b197603d73f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:14:57.198604 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b26f7c-8376-4593-b36f-b197603d73f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:14:57.202170 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b26f7c-8376-4593-b36f-b197603d73f1 service neutron] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:14:57.229889 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 811/1645] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:14:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:15:00.995603 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af2d4af-65ff-4c36-8028-6d6bde31f287 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:00.998172 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-7af2d4af-65ff-4c36-8028-6d6bde31f287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:01.021836 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af2d4af-65ff-4c36-8028-6d6bde31f287 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:01.045954 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f357fc6-7a94-4ae3-9ecf-ba9ca58dbc3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:01.046648 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f357fc6-7a94-4ae3-9ecf-ba9ca58dbc3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:01.046911 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f357fc6-7a94-4ae3-9ecf-ba9ca58dbc3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:01.047228 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f357fc6-7a94-4ae3-9ecf-ba9ca58dbc3f admin admin] PATH_INFO: `/v3/projects/0816f35db1824aa8b0b64a6459030c51` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:01.048667 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f357fc6-7a94-4ae3-9ecf-ba9ca58dbc3f admin admin] RBAC: Authorizing `identity:delete_project(project_id=0816f35db1824aa8b0b64a6459030c51)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:01.049832 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: WARNING py.warnings [None req-3f357fc6-7a94-4ae3-9ecf-ba9ca58dbc3f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 06 20:15:01.049832 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: warnings.warn(msg) Sep 06 20:15:01.049832 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: Sep 06 20:15:01.051226 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f357fc6-7a94-4ae3-9ecf-ba9ca58dbc3f admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:01.197599 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-3f357fc6-7a94-4ae3-9ecf-ba9ca58dbc3f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0816f35db1824aa8b0b64a6459030c51', 'request_id': 'req-3f357fc6-7a94-4ae3-9ecf-ba9ca58dbc3f'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:01.215819 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-3f357fc6-7a94-4ae3-9ecf-ba9ca58dbc3f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0816f35db1824aa8b0b64a6459030c51', 'request_id': 'req-3f357fc6-7a94-4ae3-9ecf-ba9ca58dbc3f'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:01.254753 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-3f357fc6-7a94-4ae3-9ecf-ba9ca58dbc3f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0816f35db1824aa8b0b64a6459030c51', 'request_id': 'req-3f357fc6-7a94-4ae3-9ecf-ba9ca58dbc3f'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:01.282261 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-3f357fc6-7a94-4ae3-9ecf-ba9ca58dbc3f admin admin] The token cache is being invalidate because project 0816f35db1824aa8b0b64a6459030c51 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81772) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:15:01.282521 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-3f357fc6-7a94-4ae3-9ecf-ba9ca58dbc3f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:01.285640 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-3f357fc6-7a94-4ae3-9ecf-ba9ca58dbc3f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:01.287565 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 835/1646] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:15:00 2022] DELETE /identity/v3/projects/0816f35db1824aa8b0b64a6459030c51 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:15:01.293450 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b26f7c-8376-4593-b36f-b197603d73f1 service neutron] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:01.313675 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b26f7c-8376-4593-b36f-b197603d73f1 service neutron] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:01.511966 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b8d283-d1ff-45da-9c73-a5a3a85cfd5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:01.512669 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b8d283-d1ff-45da-9c73-a5a3a85cfd5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:01.512956 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b8d283-d1ff-45da-9c73-a5a3a85cfd5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:01.513351 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b8d283-d1ff-45da-9c73-a5a3a85cfd5c admin admin] PATH_INFO: `/v3/users/afe77006175f4df8a164eba0330dad7c` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:01.514869 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b8d283-d1ff-45da-9c73-a5a3a85cfd5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=afe77006175f4df8a164eba0330dad7c)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:01.518604 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b8d283-d1ff-45da-9c73-a5a3a85cfd5c admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:01.614433 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:15:01.638845 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-76b8d283-d1ff-45da-9c73-a5a3a85cfd5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afe77006175f4df8a164eba0330dad7c'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:01.665955 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-76b8d283-d1ff-45da-9c73-a5a3a85cfd5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afe77006175f4df8a164eba0330dad7c'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:01.666933 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-76b8d283-d1ff-45da-9c73-a5a3a85cfd5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afe77006175f4df8a164eba0330dad7c'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:01.669577 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-76b8d283-d1ff-45da-9c73-a5a3a85cfd5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afe77006175f4df8a164eba0330dad7c'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:01.689661 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-76b8d283-d1ff-45da-9c73-a5a3a85cfd5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afe77006175f4df8a164eba0330dad7c'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:01.744037 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 812/1647] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:15:01 2022] DELETE /identity/v3/users/afe77006175f4df8a164eba0330dad7c => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:15:04.119279 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79d2d28-f1e9-452e-b738-b55adb2c0c28 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:04.120005 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79d2d28-f1e9-452e-b738-b55adb2c0c28 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:04.120423 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79d2d28-f1e9-452e-b738-b55adb2c0c28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:04.253917 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b8d283-d1ff-45da-9c73-a5a3a85cfd5c admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:04.256250 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-76b8d283-d1ff-45da-9c73-a5a3a85cfd5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:04.292779 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:15:04.311403 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-f79d2d28-f1e9-452e-b738-b55adb2c0c28 None None] MFA Rules not processed for user `ecde3e4bfd8b4f3490edd3832cea9e48`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:15:04.315454 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-f79d2d28-f1e9-452e-b738-b55adb2c0c28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:04.614417 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b8d283-d1ff-45da-9c73-a5a3a85cfd5c admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:04.635510 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cc7a86-e020-4287-b60d-b41e90a84718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:04.636821 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cc7a86-e020-4287-b60d-b41e90a84718 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:04.637309 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cc7a86-e020-4287-b60d-b41e90a84718 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:04.637787 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cc7a86-e020-4287-b60d-b41e90a84718 admin admin] PATH_INFO: `/v3/projects/64f6e0fd681c4e42a3fcc256c1123eef` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:04.639836 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cc7a86-e020-4287-b60d-b41e90a84718 admin admin] RBAC: Authorizing `identity:delete_project(project_id=64f6e0fd681c4e42a3fcc256c1123eef)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:04.642655 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cc7a86-e020-4287-b60d-b41e90a84718 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:04.698743 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 836/1648] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:15:04 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 582 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:15:04.716707 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ac9e05-046d-4e81-aba5-fd6252ae5290 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:04.717832 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ac9e05-046d-4e81-aba5-fd6252ae5290 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:04.718519 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ac9e05-046d-4e81-aba5-fd6252ae5290 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:04.815585 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-80cc7a86-e020-4287-b60d-b41e90a84718 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64f6e0fd681c4e42a3fcc256c1123eef', 'request_id': 'req-80cc7a86-e020-4287-b60d-b41e90a84718'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:04.817339 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-a6ac9e05-046d-4e81-aba5-fd6252ae5290 None None] MFA Rules not processed for user `ecde3e4bfd8b4f3490edd3832cea9e48`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:15:04.825147 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-a6ac9e05-046d-4e81-aba5-fd6252ae5290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:04.831426 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-80cc7a86-e020-4287-b60d-b41e90a84718 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64f6e0fd681c4e42a3fcc256c1123eef', 'request_id': 'req-80cc7a86-e020-4287-b60d-b41e90a84718'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:04.852348 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-80cc7a86-e020-4287-b60d-b41e90a84718 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64f6e0fd681c4e42a3fcc256c1123eef', 'request_id': 'req-80cc7a86-e020-4287-b60d-b41e90a84718'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:04.899706 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-80cc7a86-e020-4287-b60d-b41e90a84718 admin admin] The token cache is being invalidate because project 64f6e0fd681c4e42a3fcc256c1123eef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81771) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:15:04.900223 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-80cc7a86-e020-4287-b60d-b41e90a84718 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:04.901912 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-80cc7a86-e020-4287-b60d-b41e90a84718 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:04.908851 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 813/1649] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:15:04 2022] DELETE /identity/v3/projects/64f6e0fd681c4e42a3fcc256c1123eef => generated 0 bytes in 656 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:15:04.917441 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fa6f8f-e4e1-4347-a384-d9afa9b54df6 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:04.931974 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fa6f8f-e4e1-4347-a384-d9afa9b54df6 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:04.933132 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fa6f8f-e4e1-4347-a384-d9afa9b54df6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:05.008438 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-a0fa6f8f-e4e1-4347-a384-d9afa9b54df6 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:15:05.015941 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-a0fa6f8f-e4e1-4347-a384-d9afa9b54df6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:05.069511 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 837/1650] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:15:04 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 352 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:15:05.131993 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3333b64f-5eee-4355-b526-041f702b985d None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:05.133508 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3333b64f-5eee-4355-b526-041f702b985d None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:05.133508 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3333b64f-5eee-4355-b526-041f702b985d None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:05.138220 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 838/1651] 10.222.0.20 () {54 vars in 962 bytes} [Tue Sep 6 20:15:05 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:15:05.179707 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e713590-d47b-44fd-88a8-4288a1687688 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:05.180457 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e713590-d47b-44fd-88a8-4288a1687688 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:05.180639 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e713590-d47b-44fd-88a8-4288a1687688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:05.233928 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:15:05.315309 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 814/1652] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:15:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 400 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:15:05.331932 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eb8f64-fe96-4615-8387-5ec165f7ee65 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:05.332120 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eb8f64-fe96-4615-8387-5ec165f7ee65 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:05.332394 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eb8f64-fe96-4615-8387-5ec165f7ee65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:05.394154 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-e9eb8f64-fe96-4615-8387-5ec165f7ee65 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:15:05.395510 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-e9eb8f64-fe96-4615-8387-5ec165f7ee65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:05.396653 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-6e713590-d47b-44fd-88a8-4288a1687688 None None] MFA Rules not processed for user `5776cbf4f6714b2083c3ebdd6d98cc87`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:15:05.403595 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-6e713590-d47b-44fd-88a8-4288a1687688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:05.413673 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 815/1653] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:15:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:15:05.417566 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eb8f64-fe96-4615-8387-5ec165f7ee65 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:05.439343 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eb8f64-fe96-4615-8387-5ec165f7ee65 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:05.449970 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b10b8ee-2e3b-4c6f-8b72-de1a679e47b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:05.450895 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b10b8ee-2e3b-4c6f-8b72-de1a679e47b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:05.451400 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b10b8ee-2e3b-4c6f-8b72-de1a679e47b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:05.451709 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b10b8ee-2e3b-4c6f-8b72-de1a679e47b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:05.453042 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b10b8ee-2e3b-4c6f-8b72-de1a679e47b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:15:05.453403 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b10b8ee-2e3b-4c6f-8b72-de1a679e47b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:05.456810 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b10b8ee-2e3b-4c6f-8b72-de1a679e47b7 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:05.501242 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 816/1654] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 6 20:15:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:15:05.511420 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b10b8ee-2e3b-4c6f-8b72-de1a679e47b7 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:05.562145 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b10b8ee-2e3b-4c6f-8b72-de1a679e47b7 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:05.581761 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a17e83-1192-4c0a-9bb4-e7b9b4349bbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:05.582672 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a17e83-1192-4c0a-9bb4-e7b9b4349bbc admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:05.583157 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a17e83-1192-4c0a-9bb4-e7b9b4349bbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:05.583755 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a17e83-1192-4c0a-9bb4-e7b9b4349bbc admin admin] PATH_INFO: `/v3/projects` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:05.586726 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a17e83-1192-4c0a-9bb4-e7b9b4349bbc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:05.590549 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a17e83-1192-4c0a-9bb4-e7b9b4349bbc admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:05.712018 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 817/1655] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:15:05 2022] POST /identity/v3/projects => generated 381 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:15:05.722682 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a17e83-1192-4c0a-9bb4-e7b9b4349bbc admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:05.816630 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 839/1656] 10.222.0.20 () {58 vars in 1059 bytes} [Tue Sep 6 20:15:05 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 639 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:15:05.826240 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e713590-d47b-44fd-88a8-4288a1687688 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:05.858376 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e713590-d47b-44fd-88a8-4288a1687688 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:05.960480 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:15:05.982813 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a17e83-1192-4c0a-9bb4-e7b9b4349bbc admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:06.015276 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfa67f2-bd0d-4f97-8e30-4d3396e03f2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:06.016165 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfa67f2-bd0d-4f97-8e30-4d3396e03f2e admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:06.016505 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfa67f2-bd0d-4f97-8e30-4d3396e03f2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:06.016887 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfa67f2-bd0d-4f97-8e30-4d3396e03f2e admin admin] PATH_INFO: `/v3/users` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:06.017576 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfa67f2-bd0d-4f97-8e30-4d3396e03f2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:06.020359 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfa67f2-bd0d-4f97-8e30-4d3396e03f2e admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:06.065996 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 818/1657] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:15:05 2022] POST /identity/v3/users => generated 291 bytes in 352 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:15:06.075139 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfa67f2-bd0d-4f97-8e30-4d3396e03f2e admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:06.088672 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfa67f2-bd0d-4f97-8e30-4d3396e03f2e admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:06.097101 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb0b6e4-f20f-488c-80ca-82662ade5687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:06.097949 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb0b6e4-f20f-488c-80ca-82662ade5687 admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:06.098482 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb0b6e4-f20f-488c-80ca-82662ade5687 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:06.098854 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb0b6e4-f20f-488c-80ca-82662ade5687 admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:06.099728 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb0b6e4-f20f-488c-80ca-82662ade5687 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:06.099885 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52afb2a-fd7c-436b-bbe9-294f1aacc5bd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5776cbf4f6714b2083c3ebdd6d98cc87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:06.100826 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52afb2a-fd7c-436b-bbe9-294f1aacc5bd service nova] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:06.101317 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52afb2a-fd7c-436b-bbe9-294f1aacc5bd service nova] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:06.102653 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52afb2a-fd7c-436b-bbe9-294f1aacc5bd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:06.105713 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb0b6e4-f20f-488c-80ca-82662ade5687 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:06.106049 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-f52afb2a-fd7c-436b-bbe9-294f1aacc5bd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:06.141133 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 819/1658] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:15:06 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:15:06.142469 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb0b6e4-f20f-488c-80ca-82662ade5687 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:06.158753 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb0b6e4-f20f-488c-80ca-82662ade5687 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:06.184171 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d03af30-c153-4fc1-acbf-cc34492541a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:06.184803 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d03af30-c153-4fc1-acbf-cc34492541a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:06.187235 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d03af30-c153-4fc1-acbf-cc34492541a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:06.189309 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d03af30-c153-4fc1-acbf-cc34492541a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:06.189512 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d03af30-c153-4fc1-acbf-cc34492541a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:06.193376 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d03af30-c153-4fc1-acbf-cc34492541a9 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:06.228412 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 820/1659] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:15:06 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:15:06.232004 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d03af30-c153-4fc1-acbf-cc34492541a9 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:06.254791 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d03af30-c153-4fc1-acbf-cc34492541a9 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:06.267259 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f54ba7-42e5-4252-9541-523059014633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:06.281098 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f54ba7-42e5-4252-9541-523059014633 admin admin] REQUEST_METHOD: `PUT` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:06.282020 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f54ba7-42e5-4252-9541-523059014633 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:06.284262 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f54ba7-42e5-4252-9541-523059014633 admin admin] PATH_INFO: `/v3/projects/75d0baa7df6840c196563f932da7edb0/users/e59cacd832c34ec3a2bdc50b8d1f78bb/roles/4eea39e24b06431aa357053fe7907a3d` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:06.330016 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52afb2a-fd7c-436b-bbe9-294f1aacc5bd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:06.336967 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52afb2a-fd7c-436b-bbe9-294f1aacc5bd service nova] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:06.343924 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f54ba7-42e5-4252-9541-523059014633 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75d0baa7df6840c196563f932da7edb0, user_id=e59cacd832c34ec3a2bdc50b8d1f78bb, role_id=4eea39e24b06431aa357053fe7907a3d)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:06.349687 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f54ba7-42e5-4252-9541-523059014633 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:06.389943 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 821/1660] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:15:06 2022] PUT /identity/v3/projects/75d0baa7df6840c196563f932da7edb0/users/e59cacd832c34ec3a2bdc50b8d1f78bb/roles/4eea39e24b06431aa357053fe7907a3d => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:15:06.393521 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75457ebf-7702-4147-b958-3ff7219d9886 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:06.395341 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75457ebf-7702-4147-b958-3ff7219d9886 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:06.395770 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75457ebf-7702-4147-b958-3ff7219d9886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:06.408193 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 840/1661] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:15:05 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 582 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:15:06.503788 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-75457ebf-7702-4147-b958-3ff7219d9886 None None] MFA Rules not processed for user `e59cacd832c34ec3a2bdc50b8d1f78bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:15:06.505522 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-75457ebf-7702-4147-b958-3ff7219d9886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:06.654306 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:15:06.819508 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 822/1662] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:15:06 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 429 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:15:06.827179 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52afb2a-fd7c-436b-bbe9-294f1aacc5bd service nova] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:06.862424 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52afb2a-fd7c-436b-bbe9-294f1aacc5bd service nova] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:07.114300 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f7992c-1df1-4d87-bc43-89a252a9eb0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:07.118899 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f7992c-1df1-4d87-bc43-89a252a9eb0b admin admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:07.120726 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f7992c-1df1-4d87-bc43-89a252a9eb0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:07.121630 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f7992c-1df1-4d87-bc43-89a252a9eb0b admin admin] PATH_INFO: `/v3/projects` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:07.124489 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f7992c-1df1-4d87-bc43-89a252a9eb0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:07.132054 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f7992c-1df1-4d87-bc43-89a252a9eb0b admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:07.307009 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e79f04-5a29-4432-b8e3-da7b3f3c76d6 None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:07.307009 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e79f04-5a29-4432-b8e3-da7b3f3c76d6 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:07.307009 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e79f04-5a29-4432-b8e3-da7b3f3c76d6 None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:07.308969 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 823/1663] 10.222.0.20 () {54 vars in 912 bytes} [Tue Sep 6 20:15:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:15:07.314975 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37288e51-6ad1-4125-b174-8fe6ef728464 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:07.315433 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37288e51-6ad1-4125-b174-8fe6ef728464 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:07.315578 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37288e51-6ad1-4125-b174-8fe6ef728464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:07.347605 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 841/1664] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:15:06 2022] POST /identity/v3/projects => generated 383 bytes in 522 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:15:07.353424 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f7992c-1df1-4d87-bc43-89a252a9eb0b admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:07.482099 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-37288e51-6ad1-4125-b174-8fe6ef728464 None None] MFA Rules not processed for user `5776cbf4f6714b2083c3ebdd6d98cc87`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:15:07.488360 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-37288e51-6ad1-4125-b174-8fe6ef728464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:07.574846 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:15:07.637557 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f7992c-1df1-4d87-bc43-89a252a9eb0b admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:07.659929 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395c7270-31f5-4e0e-894f-34d7330c94ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:07.660600 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395c7270-31f5-4e0e-894f-34d7330c94ec admin admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:07.660899 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395c7270-31f5-4e0e-894f-34d7330c94ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:07.661647 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395c7270-31f5-4e0e-894f-34d7330c94ec admin admin] PATH_INFO: `/v3/users` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:07.663022 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395c7270-31f5-4e0e-894f-34d7330c94ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:07.666015 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395c7270-31f5-4e0e-894f-34d7330c94ec admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:07.718449 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 842/1665] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:15:07 2022] POST /identity/v3/users => generated 298 bytes in 367 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:15:07.724220 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395c7270-31f5-4e0e-894f-34d7330c94ec admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:07.757409 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395c7270-31f5-4e0e-894f-34d7330c94ec admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:07.825376 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c72594f-288a-4a6f-b8d1-e84e66e65f92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:07.830285 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c72594f-288a-4a6f-b8d1-e84e66e65f92 admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:07.870696 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c72594f-288a-4a6f-b8d1-e84e66e65f92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:07.874091 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c72594f-288a-4a6f-b8d1-e84e66e65f92 admin admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:07.875203 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c72594f-288a-4a6f-b8d1-e84e66e65f92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:07.900409 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 824/1666] 10.222.0.20 () {58 vars in 1009 bytes} [Tue Sep 6 20:15:07 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 586 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:15:07.903923 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c72594f-288a-4a6f-b8d1-e84e66e65f92 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:07.909290 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37288e51-6ad1-4125-b174-8fe6ef728464 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:07.920207 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-37288e51-6ad1-4125-b174-8fe6ef728464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:08.029872 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 843/1667] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:15:07 2022] GET /identity/v3/roles => generated 1420 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:15:08.031641 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c72594f-288a-4a6f-b8d1-e84e66e65f92 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:08.119666 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c72594f-288a-4a6f-b8d1-e84e66e65f92 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:08.146129 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce278d6-a527-4327-ab2b-90c834fd1fcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:08.152971 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce278d6-a527-4327-ab2b-90c834fd1fcd admin admin] REQUEST_METHOD: `PUT` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:08.154061 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce278d6-a527-4327-ab2b-90c834fd1fcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:08.156850 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce278d6-a527-4327-ab2b-90c834fd1fcd admin admin] PATH_INFO: `/v3/projects/101bb4a61c984a0498ff7e8bb638bb5f/users/237539c7e9ff46df8de07acf0ea200ed/roles/e0ab41d8ae22474688b5361bcc75e0f0` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:08.269933 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce278d6-a527-4327-ab2b-90c834fd1fcd admin admin] RBAC: Authorizing `identity:create_grant(project_id=101bb4a61c984a0498ff7e8bb638bb5f, user_id=237539c7e9ff46df8de07acf0ea200ed, role_id=e0ab41d8ae22474688b5361bcc75e0f0)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:08.276354 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce278d6-a527-4327-ab2b-90c834fd1fcd admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:08.344725 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37288e51-6ad1-4125-b174-8fe6ef728464 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:08.358473 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 844/1668] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:15:08 2022] PUT /identity/v3/projects/101bb4a61c984a0498ff7e8bb638bb5f/users/237539c7e9ff46df8de07acf0ea200ed/roles/e0ab41d8ae22474688b5361bcc75e0f0 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:15:08.366656 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce278d6-a527-4327-ab2b-90c834fd1fcd admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:08.713353 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6017caee-adef-4784-a625-c4781e9f1253 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07af18f35ffc42a295278d5349e06381', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:08.714620 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6017caee-adef-4784-a625-c4781e9f1253 service glance] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:08.715380 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6017caee-adef-4784-a625-c4781e9f1253 service glance] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:08.715852 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6017caee-adef-4784-a625-c4781e9f1253 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:08.820153 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce278d6-a527-4327-ab2b-90c834fd1fcd admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:08.833552 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:15:08.854270 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af530e27-1b43-45a8-a90d-a7aa7c7f52e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:08.857851 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af530e27-1b43-45a8-a90d-a7aa7c7f52e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:08.858698 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af530e27-1b43-45a8-a90d-a7aa7c7f52e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:08.861705 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af530e27-1b43-45a8-a90d-a7aa7c7f52e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:08.864223 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af530e27-1b43-45a8-a90d-a7aa7c7f52e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:08.873363 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af530e27-1b43-45a8-a90d-a7aa7c7f52e8 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:08.958103 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 845/1669] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:15:08 2022] GET /identity/v3/roles => generated 1420 bytes in 594 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:15:08.966958 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af530e27-1b43-45a8-a90d-a7aa7c7f52e8 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:09.010168 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af530e27-1b43-45a8-a90d-a7aa7c7f52e8 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:09.018017 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6017caee-adef-4784-a625-c4781e9f1253 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:09.019752 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6017caee-adef-4784-a625-c4781e9f1253 service glance] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:09.024707 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184d898a-76df-45f5-a167-6b0c7ef6d7d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:09.026414 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184d898a-76df-45f5-a167-6b0c7ef6d7d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:09.027389 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184d898a-76df-45f5-a167-6b0c7ef6d7d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:09.027771 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184d898a-76df-45f5-a167-6b0c7ef6d7d8 admin admin] PATH_INFO: `/v3/projects/101bb4a61c984a0498ff7e8bb638bb5f/users/237539c7e9ff46df8de07acf0ea200ed/roles/e0ab41d8ae22474688b5361bcc75e0f0` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:09.037520 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184d898a-76df-45f5-a167-6b0c7ef6d7d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=101bb4a61c984a0498ff7e8bb638bb5f, user_id=237539c7e9ff46df8de07acf0ea200ed, role_id=e0ab41d8ae22474688b5361bcc75e0f0)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:09.047569 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184d898a-76df-45f5-a167-6b0c7ef6d7d8 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:09.070665 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 825/1670] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:15:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 1161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 20:15:09.088939 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 846/1671] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:15:08 2022] PUT /identity/v3/projects/101bb4a61c984a0498ff7e8bb638bb5f/users/237539c7e9ff46df8de07acf0ea200ed/roles/e0ab41d8ae22474688b5361bcc75e0f0 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:15:09.106083 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01458bd5-c1d5-4d33-9aca-588ea00f68a5 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:09.106660 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01458bd5-c1d5-4d33-9aca-588ea00f68a5 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:09.106660 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01458bd5-c1d5-4d33-9aca-588ea00f68a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:09.167659 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-01458bd5-c1d5-4d33-9aca-588ea00f68a5 None None] MFA Rules not processed for user `237539c7e9ff46df8de07acf0ea200ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:15:09.169723 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-01458bd5-c1d5-4d33-9aca-588ea00f68a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:09.277700 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 826/1672] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:15:09 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:15:09.294746 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184d898a-76df-45f5-a167-6b0c7ef6d7d8 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:09.298399 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-184d898a-76df-45f5-a167-6b0c7ef6d7d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:09.452542 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184d898a-76df-45f5-a167-6b0c7ef6d7d8 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:09.458514 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b54f02d-df30-41fc-a20a-124963eb05b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df1bca993bc142908cc7cf85eabd2b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:09.459441 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b54f02d-df30-41fc-a20a-124963eb05b0 service cinder] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:09.459873 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b54f02d-df30-41fc-a20a-124963eb05b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:09.460415 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b54f02d-df30-41fc-a20a-124963eb05b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:09.510478 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b54f02d-df30-41fc-a20a-124963eb05b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:09.512540 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b54f02d-df30-41fc-a20a-124963eb05b0 service cinder] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:09.538268 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 847/1673] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:15:09 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:15:09.709363 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01458bd5-c1d5-4d33-9aca-588ea00f68a5 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:09.721715 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01458bd5-c1d5-4d33-9aca-588ea00f68a5 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:09.735371 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360d7148-2627-4637-9b6b-8995d3bcd5fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df1bca993bc142908cc7cf85eabd2b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:09.736118 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360d7148-2627-4637-9b6b-8995d3bcd5fc service cinder] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:09.736278 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360d7148-2627-4637-9b6b-8995d3bcd5fc service cinder] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:09.736529 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360d7148-2627-4637-9b6b-8995d3bcd5fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:10.021662 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360d7148-2627-4637-9b6b-8995d3bcd5fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:10.033104 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360d7148-2627-4637-9b6b-8995d3bcd5fc service cinder] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:10.079607 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 827/1674] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:15:09 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:15:10.930786 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de0f739-e603-4c7a-8b5f-12b94b0df408 None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:10.935810 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de0f739-e603-4c7a-8b5f-12b94b0df408 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:10.936358 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de0f739-e603-4c7a-8b5f-12b94b0df408 None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:10.940073 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 848/1675] 10.222.0.20 () {54 vars in 912 bytes} [Tue Sep 6 20:15:10 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:15:10.950456 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b354f326-32da-41ad-b8b5-cce0978ef24f None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:10.951030 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b354f326-32da-41ad-b8b5-cce0978ef24f None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:10.951375 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b354f326-32da-41ad-b8b5-cce0978ef24f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:11.036192 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de0f739-e603-4c7a-8b5f-12b94b0df408 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:11.039915 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-2de0f739-e603-4c7a-8b5f-12b94b0df408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:11.062963 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-b354f326-32da-41ad-b8b5-cce0978ef24f None None] MFA Rules not processed for user `32882d38295046fea004a659a71bde58`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:15:11.066172 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-b354f326-32da-41ad-b8b5-cce0978ef24f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:11.092027 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:15:11.179768 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:15:11.234165 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 828/1676] 10.222.0.20 () {58 vars in 1009 bytes} [Tue Sep 6 20:15:10 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:15:11.314249 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b354f326-32da-41ad-b8b5-cce0978ef24f None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:11.317252 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-b354f326-32da-41ad-b8b5-cce0978ef24f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:11.335027 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b354f326-32da-41ad-b8b5-cce0978ef24f None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:11.335211 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de0f739-e603-4c7a-8b5f-12b94b0df408 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:11.341448 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03615431-8e1c-4bec-882f-a47914f7d4c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:11.342371 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03615431-8e1c-4bec-882f-a47914f7d4c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:11.343040 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03615431-8e1c-4bec-882f-a47914f7d4c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:11.343324 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03615431-8e1c-4bec-882f-a47914f7d4c2 admin admin] PATH_INFO: `/v3/users/4fe3ec5734d54682ab918a4dfcaa3880` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:11.344904 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03615431-8e1c-4bec-882f-a47914f7d4c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4fe3ec5734d54682ab918a4dfcaa3880)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:11.347657 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414c36d6-1578-4e27-9072-f40b013c1f47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:11.348085 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03615431-8e1c-4bec-882f-a47914f7d4c2 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:11.348554 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414c36d6-1578-4e27-9072-f40b013c1f47 service neutron] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:11.348941 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414c36d6-1578-4e27-9072-f40b013c1f47 service neutron] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:11.349360 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414c36d6-1578-4e27-9072-f40b013c1f47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:11.365106 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414c36d6-1578-4e27-9072-f40b013c1f47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:11.369638 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414c36d6-1578-4e27-9072-f40b013c1f47 service neutron] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:11.396263 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 829/1677] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:15:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:15:11.397873 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414c36d6-1578-4e27-9072-f40b013c1f47 service neutron] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:11.429447 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414c36d6-1578-4e27-9072-f40b013c1f47 service neutron] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:11.450130 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd61a87-356b-4237-8a73-93d18a9d9a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:11.451500 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd61a87-356b-4237-8a73-93d18a9d9a00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:11.453019 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd61a87-356b-4237-8a73-93d18a9d9a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:11.453796 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd61a87-356b-4237-8a73-93d18a9d9a00 admin admin] PATH_INFO: `/v3/users/e59cacd832c34ec3a2bdc50b8d1f78bb` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:11.458080 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd61a87-356b-4237-8a73-93d18a9d9a00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e59cacd832c34ec3a2bdc50b8d1f78bb)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:11.463354 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd61a87-356b-4237-8a73-93d18a9d9a00 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:11.464096 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-03615431-8e1c-4bec-882f-a47914f7d4c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4fe3ec5734d54682ab918a4dfcaa3880'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:11.467704 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-03615431-8e1c-4bec-882f-a47914f7d4c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4fe3ec5734d54682ab918a4dfcaa3880'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:11.501949 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-03615431-8e1c-4bec-882f-a47914f7d4c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4fe3ec5734d54682ab918a4dfcaa3880'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:11.502978 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-03615431-8e1c-4bec-882f-a47914f7d4c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4fe3ec5734d54682ab918a4dfcaa3880'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:11.516795 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-03615431-8e1c-4bec-882f-a47914f7d4c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4fe3ec5734d54682ab918a4dfcaa3880'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:11.558951 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 849/1678] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:15:11 2022] DELETE /identity/v3/users/4fe3ec5734d54682ab918a4dfcaa3880 => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:15:11.567464 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64320064-7093-4cfa-ac45-5825910d2800 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:11.567887 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64320064-7093-4cfa-ac45-5825910d2800 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:11.568464 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64320064-7093-4cfa-ac45-5825910d2800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:11.604633 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-64320064-7093-4cfa-ac45-5825910d2800 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:15:11.608134 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-64320064-7093-4cfa-ac45-5825910d2800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:11.678170 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-0dd61a87-356b-4237-8a73-93d18a9d9a00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e59cacd832c34ec3a2bdc50b8d1f78bb'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:11.702942 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 850/1679] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:15:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:15:11.725336 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-0dd61a87-356b-4237-8a73-93d18a9d9a00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e59cacd832c34ec3a2bdc50b8d1f78bb'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:11.728744 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-0dd61a87-356b-4237-8a73-93d18a9d9a00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e59cacd832c34ec3a2bdc50b8d1f78bb'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:11.732695 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-0dd61a87-356b-4237-8a73-93d18a9d9a00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e59cacd832c34ec3a2bdc50b8d1f78bb'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:11.760002 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-0dd61a87-356b-4237-8a73-93d18a9d9a00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e59cacd832c34ec3a2bdc50b8d1f78bb'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:11.834095 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 830/1680] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:15:11 2022] DELETE /identity/v3/users/e59cacd832c34ec3a2bdc50b8d1f78bb => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:15:11.834515 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecd8bf0-cf24-4026-9d9d-4c7646dd20b9 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:11.834798 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecd8bf0-cf24-4026-9d9d-4c7646dd20b9 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:11.835219 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecd8bf0-cf24-4026-9d9d-4c7646dd20b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:11.863100 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd61a87-356b-4237-8a73-93d18a9d9a00 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:11.867018 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-0dd61a87-356b-4237-8a73-93d18a9d9a00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:11.888048 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-1ecd8bf0-cf24-4026-9d9d-4c7646dd20b9 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:15:11.889497 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-1ecd8bf0-cf24-4026-9d9d-4c7646dd20b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:12.058452 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd61a87-356b-4237-8a73-93d18a9d9a00 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:12.066649 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10887623-1e24-495b-aa13-d9879758d79b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:12.067385 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10887623-1e24-495b-aa13-d9879758d79b service neutron] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:12.067747 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10887623-1e24-495b-aa13-d9879758d79b service neutron] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:12.068120 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10887623-1e24-495b-aa13-d9879758d79b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:12.071385 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-10887623-1e24-495b-aa13-d9879758d79b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:12.078936 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:15:12.128725 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:15:12.136941 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 851/1681] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:15:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:15:12.228580 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10887623-1e24-495b-aa13-d9879758d79b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:12.233435 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10887623-1e24-495b-aa13-d9879758d79b service neutron] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:12.267523 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 831/1682] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:15:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:15:12.316768 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecd8bf0-cf24-4026-9d9d-4c7646dd20b9 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:12.331554 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecd8bf0-cf24-4026-9d9d-4c7646dd20b9 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:12.344167 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dcdb4d-a83f-43ed-9ad1-b2d1e277e74a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:12.344827 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dcdb4d-a83f-43ed-9ad1-b2d1e277e74a service neutron] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:12.345096 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dcdb4d-a83f-43ed-9ad1-b2d1e277e74a service neutron] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:12.345400 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dcdb4d-a83f-43ed-9ad1-b2d1e277e74a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:12.370974 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dcdb4d-a83f-43ed-9ad1-b2d1e277e74a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:12.372530 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dcdb4d-a83f-43ed-9ad1-b2d1e277e74a service neutron] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:12.404055 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 852/1683] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:15:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:15:13.694488 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cf1266-0578-43b8-b231-894a9b202996 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:13.696044 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cf1266-0578-43b8-b231-894a9b202996 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:13.696498 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cf1266-0578-43b8-b231-894a9b202996 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:13.802239 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-f5cf1266-0578-43b8-b231-894a9b202996 None None] MFA Rules not processed for user `f49ef54a38614cb880577626e65b0e32`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:15:13.805746 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-f5cf1266-0578-43b8-b231-894a9b202996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:13.929564 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dcdb4d-a83f-43ed-9ad1-b2d1e277e74a service neutron] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:13.934546 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-26dcdb4d-a83f-43ed-9ad1-b2d1e277e74a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:14.169698 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 832/1684] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:15:13 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 468 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:15:14.179918 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cf1266-0578-43b8-b231-894a9b202996 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:14.180524 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dcdb4d-a83f-43ed-9ad1-b2d1e277e74a service neutron] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:14.181051 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-f5cf1266-0578-43b8-b231-894a9b202996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:14.198371 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b839dd5c-789e-4b19-89d3-94a745cc4a4d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f57872de7bb74ea48d30e93e51830aba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:14.199686 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b839dd5c-789e-4b19-89d3-94a745cc4a4d service placement] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:14.200216 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b839dd5c-789e-4b19-89d3-94a745cc4a4d service placement] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:14.200794 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b839dd5c-789e-4b19-89d3-94a745cc4a4d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:14.208788 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-b839dd5c-789e-4b19-89d3-94a745cc4a4d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:14.238621 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b839dd5c-789e-4b19-89d3-94a745cc4a4d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:14.240795 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b839dd5c-789e-4b19-89d3-94a745cc4a4d service placement] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:14.267053 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 853/1685] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:15:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:15:14.676260 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cf1266-0578-43b8-b231-894a9b202996 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:14.705265 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b8dc85-26b1-4073-a123-d713dfed3405 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5776cbf4f6714b2083c3ebdd6d98cc87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:14.707470 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b8dc85-26b1-4073-a123-d713dfed3405 service nova] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:14.712099 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b8dc85-26b1-4073-a123-d713dfed3405 service nova] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:14.716694 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b8dc85-26b1-4073-a123-d713dfed3405 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:14.762552 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b8dc85-26b1-4073-a123-d713dfed3405 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:14.780407 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b8dc85-26b1-4073-a123-d713dfed3405 service nova] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:14.865168 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 833/1686] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:15:14 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 686 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 20:15:15.005071 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df51bdc3-d31e-4e68-8882-a4b2eed80a73 None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:15.005888 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df51bdc3-d31e-4e68-8882-a4b2eed80a73 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:15.005888 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df51bdc3-d31e-4e68-8882-a4b2eed80a73 None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:15.009219 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 854/1687] 10.222.0.20 () {54 vars in 962 bytes} [Tue Sep 6 20:15:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 20:15:15.030728 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47260403-e426-483f-a186-7fc025298dfc None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:15.033100 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47260403-e426-483f-a186-7fc025298dfc None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:15.033299 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47260403-e426-483f-a186-7fc025298dfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:15.145846 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-47260403-e426-483f-a186-7fc025298dfc None None] MFA Rules not processed for user `5776cbf4f6714b2083c3ebdd6d98cc87`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:15:15.148064 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-47260403-e426-483f-a186-7fc025298dfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:15.194238 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 834/1688] 10.222.0.20 () {58 vars in 1059 bytes} [Tue Sep 6 20:15:15 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:15:15.201516 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf672675-4238-417e-be3c-2418ca4ceacc None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:15.202001 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf672675-4238-417e-be3c-2418ca4ceacc None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:15.202348 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf672675-4238-417e-be3c-2418ca4ceacc None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:15.205886 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 855/1689] 10.222.0.20 () {54 vars in 912 bytes} [Tue Sep 6 20:15:15 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:15:15.212205 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8309b0-ae37-455c-9724-5b081dc925d2 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:15.212790 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8309b0-ae37-455c-9724-5b081dc925d2 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:15.213284 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8309b0-ae37-455c-9724-5b081dc925d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:15.288039 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf672675-4238-417e-be3c-2418ca4ceacc None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:15.290109 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-bf672675-4238-417e-be3c-2418ca4ceacc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:15.295296 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-cf8309b0-ae37-455c-9724-5b081dc925d2 None None] MFA Rules not processed for user `5776cbf4f6714b2083c3ebdd6d98cc87`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:15:15.297226 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-cf8309b0-ae37-455c-9724-5b081dc925d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:15.338935 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf672675-4238-417e-be3c-2418ca4ceacc None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:15.348595 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 835/1690] 10.222.0.20 () {58 vars in 1009 bytes} [Tue Sep 6 20:15:15 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:15:15.360418 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d572e32-0b9f-4d6c-9219-db5ee3824045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:15.361098 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d572e32-0b9f-4d6c-9219-db5ee3824045 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:15.361385 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d572e32-0b9f-4d6c-9219-db5ee3824045 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:15.361698 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d572e32-0b9f-4d6c-9219-db5ee3824045 admin admin] PATH_INFO: `/v3/projects/9492a3a2d2d74a1eafee466d4e060c39` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:15.370837 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d572e32-0b9f-4d6c-9219-db5ee3824045 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9492a3a2d2d74a1eafee466d4e060c39)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:15.373781 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d572e32-0b9f-4d6c-9219-db5ee3824045 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:15.442848 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8309b0-ae37-455c-9724-5b081dc925d2 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:15.458864 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8309b0-ae37-455c-9724-5b081dc925d2 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:15.478618 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaebe53-0a8d-4118-bdce-34b8f74efdb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:15.479287 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaebe53-0a8d-4118-bdce-34b8f74efdb0 service neutron] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:15.480185 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaebe53-0a8d-4118-bdce-34b8f74efdb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:15.480487 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaebe53-0a8d-4118-bdce-34b8f74efdb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:15.512937 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaebe53-0a8d-4118-bdce-34b8f74efdb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:15.517820 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaebe53-0a8d-4118-bdce-34b8f74efdb0 service neutron] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:15.572270 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 836/1691] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:15:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 20:15:15.621127 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-1d572e32-0b9f-4d6c-9219-db5ee3824045 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9492a3a2d2d74a1eafee466d4e060c39', 'request_id': 'req-1d572e32-0b9f-4d6c-9219-db5ee3824045'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:15.652211 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-1d572e32-0b9f-4d6c-9219-db5ee3824045 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9492a3a2d2d74a1eafee466d4e060c39', 'request_id': 'req-1d572e32-0b9f-4d6c-9219-db5ee3824045'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:15.704193 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-1d572e32-0b9f-4d6c-9219-db5ee3824045 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9492a3a2d2d74a1eafee466d4e060c39', 'request_id': 'req-1d572e32-0b9f-4d6c-9219-db5ee3824045'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:15.759015 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-1d572e32-0b9f-4d6c-9219-db5ee3824045 admin admin] The token cache is being invalidate because project 9492a3a2d2d74a1eafee466d4e060c39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81772) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:15:15.759530 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-1d572e32-0b9f-4d6c-9219-db5ee3824045 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:15.766167 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-1d572e32-0b9f-4d6c-9219-db5ee3824045 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:15.776674 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 856/1692] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:15:15 2022] DELETE /identity/v3/projects/9492a3a2d2d74a1eafee466d4e060c39 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:15:15.779354 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaebe53-0a8d-4118-bdce-34b8f74efdb0 service neutron] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:15.805021 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaebe53-0a8d-4118-bdce-34b8f74efdb0 service neutron] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:15.906068 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:15:15.923005 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4b11f6-24d6-47df-a8c7-47f5c77efca9 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:15.923633 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4b11f6-24d6-47df-a8c7-47f5c77efca9 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:15.924087 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4b11f6-24d6-47df-a8c7-47f5c77efca9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:16.000058 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-cc4b11f6-24d6-47df-a8c7-47f5c77efca9 None None] MFA Rules not processed for user `ecde3e4bfd8b4f3490edd3832cea9e48`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:15:16.004690 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-cc4b11f6-24d6-47df-a8c7-47f5c77efca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:16.216845 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8b7572-911f-468b-b055-706224718845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:16.221187 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8b7572-911f-468b-b055-706224718845 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:16.221692 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8b7572-911f-468b-b055-706224718845 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:16.222192 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8b7572-911f-468b-b055-706224718845 admin admin] PATH_INFO: `/v3/users/202c3c30109f47fc9236ae3741cad37c` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:16.224019 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:15:16.238838 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8b7572-911f-468b-b055-706224718845 admin admin] RBAC: Authorizing `identity:delete_user(user_id=202c3c30109f47fc9236ae3741cad37c)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:16.258334 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8b7572-911f-468b-b055-706224718845 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:16.291135 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 857/1693] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:15:15 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 369 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:15:16.365226 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4b11f6-24d6-47df-a8c7-47f5c77efca9 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:16.367058 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-cc4b11f6-24d6-47df-a8c7-47f5c77efca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:16.424715 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4b11f6-24d6-47df-a8c7-47f5c77efca9 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:16.439023 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21aa5b3-dc33-4d30-aa50-0835fbfac2ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:16.439766 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21aa5b3-dc33-4d30-aa50-0835fbfac2ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:16.440335 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21aa5b3-dc33-4d30-aa50-0835fbfac2ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:16.440624 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21aa5b3-dc33-4d30-aa50-0835fbfac2ba admin admin] PATH_INFO: `/v3/projects/75d0baa7df6840c196563f932da7edb0` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:16.451934 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-5d8b7572-911f-468b-b055-706224718845 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '202c3c30109f47fc9236ae3741cad37c'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:16.455166 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21aa5b3-dc33-4d30-aa50-0835fbfac2ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=75d0baa7df6840c196563f932da7edb0)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:16.457202 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21aa5b3-dc33-4d30-aa50-0835fbfac2ba admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:16.500723 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-5d8b7572-911f-468b-b055-706224718845 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '202c3c30109f47fc9236ae3741cad37c'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:16.501459 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-5d8b7572-911f-468b-b055-706224718845 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '202c3c30109f47fc9236ae3741cad37c'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:16.502223 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-5d8b7572-911f-468b-b055-706224718845 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '202c3c30109f47fc9236ae3741cad37c'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:16.536544 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-5d8b7572-911f-468b-b055-706224718845 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '202c3c30109f47fc9236ae3741cad37c'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:16.629095 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 837/1694] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:15:15 2022] DELETE /identity/v3/users/202c3c30109f47fc9236ae3741cad37c => generated 0 bytes in 850 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:15:16.660470 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-b21aa5b3-dc33-4d30-aa50-0835fbfac2ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75d0baa7df6840c196563f932da7edb0', 'request_id': 'req-b21aa5b3-dc33-4d30-aa50-0835fbfac2ba'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:16.701032 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-b21aa5b3-dc33-4d30-aa50-0835fbfac2ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75d0baa7df6840c196563f932da7edb0', 'request_id': 'req-b21aa5b3-dc33-4d30-aa50-0835fbfac2ba'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:16.738295 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-b21aa5b3-dc33-4d30-aa50-0835fbfac2ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75d0baa7df6840c196563f932da7edb0', 'request_id': 'req-b21aa5b3-dc33-4d30-aa50-0835fbfac2ba'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:16.799987 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-b21aa5b3-dc33-4d30-aa50-0835fbfac2ba admin admin] The token cache is being invalidate because project 75d0baa7df6840c196563f932da7edb0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81772) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:15:16.801859 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-b21aa5b3-dc33-4d30-aa50-0835fbfac2ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:16.812979 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-b21aa5b3-dc33-4d30-aa50-0835fbfac2ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:16.829059 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 858/1695] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:15:16 2022] DELETE /identity/v3/projects/75d0baa7df6840c196563f932da7edb0 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:15:16.830124 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8b7572-911f-468b-b055-706224718845 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:16.832454 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-5d8b7572-911f-468b-b055-706224718845 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:17.022316 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8b7572-911f-468b-b055-706224718845 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:17.041461 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2bbfb0-cf28-4344-90c7-c2acc625529c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:17.042605 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2bbfb0-cf28-4344-90c7-c2acc625529c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:17.043162 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2bbfb0-cf28-4344-90c7-c2acc625529c admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:17.043744 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2bbfb0-cf28-4344-90c7-c2acc625529c admin admin] PATH_INFO: `/v3/users/237539c7e9ff46df8de07acf0ea200ed` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:17.045790 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2bbfb0-cf28-4344-90c7-c2acc625529c admin admin] RBAC: Authorizing `identity:delete_user(user_id=237539c7e9ff46df8de07acf0ea200ed)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:17.048068 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2bbfb0-cf28-4344-90c7-c2acc625529c admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:17.155309 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:15:17.165889 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-ae2bbfb0-cf28-4344-90c7-c2acc625529c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '237539c7e9ff46df8de07acf0ea200ed'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:17.193294 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-ae2bbfb0-cf28-4344-90c7-c2acc625529c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '237539c7e9ff46df8de07acf0ea200ed'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:17.195447 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-ae2bbfb0-cf28-4344-90c7-c2acc625529c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '237539c7e9ff46df8de07acf0ea200ed'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:17.199039 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-ae2bbfb0-cf28-4344-90c7-c2acc625529c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '237539c7e9ff46df8de07acf0ea200ed'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:17.219755 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-ae2bbfb0-cf28-4344-90c7-c2acc625529c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '237539c7e9ff46df8de07acf0ea200ed'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:17.262595 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 838/1696] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:15:16 2022] DELETE /identity/v3/users/237539c7e9ff46df8de07acf0ea200ed => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:15:19.132805 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21aa5b3-dc33-4d30-aa50-0835fbfac2ba admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:19.137110 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-b21aa5b3-dc33-4d30-aa50-0835fbfac2ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:19.378685 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21aa5b3-dc33-4d30-aa50-0835fbfac2ba admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:19.608081 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a80eb9-9180-484e-b344-558f14c54466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:19.608837 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a80eb9-9180-484e-b344-558f14c54466 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:19.609124 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a80eb9-9180-484e-b344-558f14c54466 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:19.609406 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a80eb9-9180-484e-b344-558f14c54466 admin admin] PATH_INFO: `/v3/projects/572acb575fed41879328d05805ae264e` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:19.620680 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a80eb9-9180-484e-b344-558f14c54466 admin admin] RBAC: Authorizing `identity:delete_project(project_id=572acb575fed41879328d05805ae264e)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:19.628496 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a80eb9-9180-484e-b344-558f14c54466 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:19.755854 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:15:19.805438 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-21a80eb9-9180-484e-b344-558f14c54466 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '572acb575fed41879328d05805ae264e', 'request_id': 'req-21a80eb9-9180-484e-b344-558f14c54466'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:19.826659 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-21a80eb9-9180-484e-b344-558f14c54466 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '572acb575fed41879328d05805ae264e', 'request_id': 'req-21a80eb9-9180-484e-b344-558f14c54466'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:19.852518 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-21a80eb9-9180-484e-b344-558f14c54466 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '572acb575fed41879328d05805ae264e', 'request_id': 'req-21a80eb9-9180-484e-b344-558f14c54466'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:19.940710 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-21a80eb9-9180-484e-b344-558f14c54466 admin admin] The token cache is being invalidate because project 572acb575fed41879328d05805ae264e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81772) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:15:19.943702 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-21a80eb9-9180-484e-b344-558f14c54466 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:19.947264 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-21a80eb9-9180-484e-b344-558f14c54466 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:19.964930 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 859/1697] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:15:19 2022] DELETE /identity/v3/projects/572acb575fed41879328d05805ae264e => generated 0 bytes in 833 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:15:19.970457 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3270e2ce-43d1-4c8f-9c10-13c0913a160a None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:19.972005 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3270e2ce-43d1-4c8f-9c10-13c0913a160a None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:19.972775 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3270e2ce-43d1-4c8f-9c10-13c0913a160a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:20.066233 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-3270e2ce-43d1-4c8f-9c10-13c0913a160a None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:15:20.068490 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-3270e2ce-43d1-4c8f-9c10-13c0913a160a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:20.235301 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a62b5e-47b5-4241-b2aa-b2c6f087634a None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:20.236458 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a62b5e-47b5-4241-b2aa-b2c6f087634a None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:20.237304 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a62b5e-47b5-4241-b2aa-b2c6f087634a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:20.293222 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 839/1698] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:15:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:15:20.305033 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0033763-ce58-40af-a245-4c814f8d789a None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:20.305559 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0033763-ce58-40af-a245-4c814f8d789a None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:20.306151 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0033763-ce58-40af-a245-4c814f8d789a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:20.439623 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-e0033763-ce58-40af-a245-4c814f8d789a None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:15:20.442335 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-e0033763-ce58-40af-a245-4c814f8d789a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:20.482999 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 840/1699] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:15:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:15:20.492917 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0033763-ce58-40af-a245-4c814f8d789a None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:20.510674 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-a1a62b5e-47b5-4241-b2aa-b2c6f087634a None None] MFA Rules not processed for user `1a52117df70447dba9bbee35d083e44f`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:15:20.517722 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-a1a62b5e-47b5-4241-b2aa-b2c6f087634a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:20.566735 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0033763-ce58-40af-a245-4c814f8d789a None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:20.582926 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df54bd50-ce2b-4d31-b504-09ed7fbfd08d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:20.583613 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df54bd50-ce2b-4d31-b504-09ed7fbfd08d admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:20.583898 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df54bd50-ce2b-4d31-b504-09ed7fbfd08d admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:20.584169 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df54bd50-ce2b-4d31-b504-09ed7fbfd08d admin admin] PATH_INFO: `/v3/domains` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:20.585511 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df54bd50-ce2b-4d31-b504-09ed7fbfd08d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:15:20.585844 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df54bd50-ce2b-4d31-b504-09ed7fbfd08d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:20.587576 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df54bd50-ce2b-4d31-b504-09ed7fbfd08d admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:20.656441 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 841/1700] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 6 20:15:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:15:20.664750 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df54bd50-ce2b-4d31-b504-09ed7fbfd08d admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:20.692262 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df54bd50-ce2b-4d31-b504-09ed7fbfd08d admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:20.714108 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1d17d6-6aba-4b56-894e-9a04bb606e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:20.715282 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1d17d6-6aba-4b56-894e-9a04bb606e60 admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:20.715282 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1d17d6-6aba-4b56-894e-9a04bb606e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:20.715461 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1d17d6-6aba-4b56-894e-9a04bb606e60 admin admin] PATH_INFO: `/v3/projects` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:20.716021 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1d17d6-6aba-4b56-894e-9a04bb606e60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:20.718448 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1d17d6-6aba-4b56-894e-9a04bb606e60 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:20.760013 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 860/1701] 10.222.0.20 () {58 vars in 1009 bytes} [Tue Sep 6 20:15:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 528 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:15:20.807649 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 842/1702] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:15:20 2022] POST /identity/v3/projects => generated 365 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:15:20.813021 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a62b5e-47b5-4241-b2aa-b2c6f087634a None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:20.860743 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a62b5e-47b5-4241-b2aa-b2c6f087634a None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:21.049371 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dda57d-39b3-414d-ac50-e9b92465ae28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:21.050331 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dda57d-39b3-414d-ac50-e9b92465ae28 admin admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:21.052416 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dda57d-39b3-414d-ac50-e9b92465ae28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:21.054333 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dda57d-39b3-414d-ac50-e9b92465ae28 admin admin] PATH_INFO: `/v3/users` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:21.056724 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dda57d-39b3-414d-ac50-e9b92465ae28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:21.072280 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dda57d-39b3-414d-ac50-e9b92465ae28 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:21.077480 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1d17d6-6aba-4b56-894e-9a04bb606e60 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:21.079566 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-fa1d17d6-6aba-4b56-894e-9a04bb606e60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:21.147833 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 861/1703] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:15:20 2022] POST /identity/v3/users => generated 283 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:15:21.153137 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dda57d-39b3-414d-ac50-e9b92465ae28 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:21.191977 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dda57d-39b3-414d-ac50-e9b92465ae28 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:21.206581 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56b0da5-ff51-4e82-8237-11eafc5ecef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:21.207273 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56b0da5-ff51-4e82-8237-11eafc5ecef0 admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:21.207565 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56b0da5-ff51-4e82-8237-11eafc5ecef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:21.207843 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56b0da5-ff51-4e82-8237-11eafc5ecef0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:21.208700 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56b0da5-ff51-4e82-8237-11eafc5ecef0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:21.211188 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56b0da5-ff51-4e82-8237-11eafc5ecef0 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:21.250075 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 862/1704] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:15:21 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:15:21.255151 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56b0da5-ff51-4e82-8237-11eafc5ecef0 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:21.264340 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:15:21.279175 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56b0da5-ff51-4e82-8237-11eafc5ecef0 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:21.294874 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327c3a7c-8849-4f3a-b4a2-6e6371220f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:21.294874 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327c3a7c-8849-4f3a-b4a2-6e6371220f0b admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:21.294874 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327c3a7c-8849-4f3a-b4a2-6e6371220f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:21.294874 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327c3a7c-8849-4f3a-b4a2-6e6371220f0b admin admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:21.295281 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327c3a7c-8849-4f3a-b4a2-6e6371220f0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:21.318483 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327c3a7c-8849-4f3a-b4a2-6e6371220f0b admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:21.368829 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 863/1705] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:15:21 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 20:15:21.373090 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327c3a7c-8849-4f3a-b4a2-6e6371220f0b admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:21.396818 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:15:21.402451 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327c3a7c-8849-4f3a-b4a2-6e6371220f0b admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:21.429713 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d07ec4-0f7b-4fcf-9946-0b718931440b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:21.433041 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d07ec4-0f7b-4fcf-9946-0b718931440b admin admin] REQUEST_METHOD: `PUT` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:21.433338 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d07ec4-0f7b-4fcf-9946-0b718931440b admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:21.433676 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d07ec4-0f7b-4fcf-9946-0b718931440b admin admin] PATH_INFO: `/v3/projects/44654764d56041cda9b28cd3f6953729/users/50f1911c4a5743aeb1c1ee91c524c9b7/roles/4eea39e24b06431aa357053fe7907a3d` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:21.444847 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1d17d6-6aba-4b56-894e-9a04bb606e60 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:21.476266 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a876d3-778e-4b33-a29b-bdcd3b420373 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b72516236da47e292d78de201b0552b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:21.480650 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a876d3-778e-4b33-a29b-bdcd3b420373 service swift] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:21.480955 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a876d3-778e-4b33-a29b-bdcd3b420373 service swift] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:21.481242 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a876d3-778e-4b33-a29b-bdcd3b420373 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:21.508549 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d07ec4-0f7b-4fcf-9946-0b718931440b admin admin] RBAC: Authorizing `identity:create_grant(project_id=44654764d56041cda9b28cd3f6953729, user_id=50f1911c4a5743aeb1c1ee91c524c9b7, role_id=4eea39e24b06431aa357053fe7907a3d)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:21.514254 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d07ec4-0f7b-4fcf-9946-0b718931440b admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:21.565657 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 864/1706] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:15:21 2022] PUT /identity/v3/projects/44654764d56041cda9b28cd3f6953729/users/50f1911c4a5743aeb1c1ee91c524c9b7/roles/4eea39e24b06431aa357053fe7907a3d => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:15:21.568076 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06be747-9f22-402e-8df4-f2645330fc6c None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:21.568372 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06be747-9f22-402e-8df4-f2645330fc6c None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:21.568652 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06be747-9f22-402e-8df4-f2645330fc6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:21.708161 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-d06be747-9f22-402e-8df4-f2645330fc6c None None] MFA Rules not processed for user `50f1911c4a5743aeb1c1ee91c524c9b7`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:15:21.709454 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-d06be747-9f22-402e-8df4-f2645330fc6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:21.740143 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a876d3-778e-4b33-a29b-bdcd3b420373 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:21.742341 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a876d3-778e-4b33-a29b-bdcd3b420373 service swift] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:21.767386 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 843/1707] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Sep 6 20:15:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 691 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 06 20:15:22.048435 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 865/1708] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:15:21 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 479 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:15:22.051192 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a876d3-778e-4b33-a29b-bdcd3b420373 service swift] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:22.085101 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a876d3-778e-4b33-a29b-bdcd3b420373 service swift] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:22.327277 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06be747-9f22-402e-8df4-f2645330fc6c None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:22.330349 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-d06be747-9f22-402e-8df4-f2645330fc6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:22.517739 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b09a3c-84af-4077-a6d8-8c43547fa1e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:22.520361 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b09a3c-84af-4077-a6d8-8c43547fa1e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:22.520599 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b09a3c-84af-4077-a6d8-8c43547fa1e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:22.522245 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b09a3c-84af-4077-a6d8-8c43547fa1e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:22.524325 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b09a3c-84af-4077-a6d8-8c43547fa1e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:22.535336 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b09a3c-84af-4077-a6d8-8c43547fa1e9 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:22.716469 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 844/1709] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:15:22 2022] POST /identity/v3/projects => generated 365 bytes in 666 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:15:22.718296 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b09a3c-84af-4077-a6d8-8c43547fa1e9 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:22.725440 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-b1b09a3c-84af-4077-a6d8-8c43547fa1e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:22.783466 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06be747-9f22-402e-8df4-f2645330fc6c None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:23.047954 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1af7a0-b951-4328-8b60-ef4a041df598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:23.049236 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1af7a0-b951-4328-8b60-ef4a041df598 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:23.051941 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1af7a0-b951-4328-8b60-ef4a041df598 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:23.052590 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1af7a0-b951-4328-8b60-ef4a041df598 admin admin] PATH_INFO: `/v3/users/02c4c24e5f2849fd975dc354511510f2` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:23.066373 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1af7a0-b951-4328-8b60-ef4a041df598 admin admin] RBAC: Authorizing `identity:delete_user(user_id=02c4c24e5f2849fd975dc354511510f2)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:23.073743 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1af7a0-b951-4328-8b60-ef4a041df598 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:23.130233 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:15:23.181020 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b09a3c-84af-4077-a6d8-8c43547fa1e9 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:23.241825 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deb11c6-0b21-42b5-9368-aff6b3be878d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:23.244553 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deb11c6-0b21-42b5-9368-aff6b3be878d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:23.246657 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deb11c6-0b21-42b5-9368-aff6b3be878d admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:23.247062 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deb11c6-0b21-42b5-9368-aff6b3be878d admin admin] PATH_INFO: `/v3/projects/101bb4a61c984a0498ff7e8bb638bb5f` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:23.253552 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4deb11c6-0b21-42b5-9368-aff6b3be878d admin admin] RBAC: Authorizing `identity:delete_project(project_id=101bb4a61c984a0498ff7e8bb638bb5f)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:23.259304 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4deb11c6-0b21-42b5-9368-aff6b3be878d admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:23.441175 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:15:23.476462 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-bc1af7a0-b951-4328-8b60-ef4a041df598 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02c4c24e5f2849fd975dc354511510f2'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:23.478506 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-bc1af7a0-b951-4328-8b60-ef4a041df598 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02c4c24e5f2849fd975dc354511510f2'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:23.526141 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-4deb11c6-0b21-42b5-9368-aff6b3be878d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '101bb4a61c984a0498ff7e8bb638bb5f', 'request_id': 'req-4deb11c6-0b21-42b5-9368-aff6b3be878d'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:23.547228 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-4deb11c6-0b21-42b5-9368-aff6b3be878d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '101bb4a61c984a0498ff7e8bb638bb5f', 'request_id': 'req-4deb11c6-0b21-42b5-9368-aff6b3be878d'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:23.547645 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-bc1af7a0-b951-4328-8b60-ef4a041df598 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02c4c24e5f2849fd975dc354511510f2'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:23.548942 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-bc1af7a0-b951-4328-8b60-ef4a041df598 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02c4c24e5f2849fd975dc354511510f2'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:23.586914 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-bc1af7a0-b951-4328-8b60-ef4a041df598 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02c4c24e5f2849fd975dc354511510f2'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:23.598619 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-4deb11c6-0b21-42b5-9368-aff6b3be878d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '101bb4a61c984a0498ff7e8bb638bb5f', 'request_id': 'req-4deb11c6-0b21-42b5-9368-aff6b3be878d'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:23.649271 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 866/1710] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:15:22 2022] DELETE /identity/v3/users/02c4c24e5f2849fd975dc354511510f2 => generated 0 bytes in 1324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 20:15:23.651102 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-4deb11c6-0b21-42b5-9368-aff6b3be878d admin admin] The token cache is being invalidate because project 101bb4a61c984a0498ff7e8bb638bb5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81771) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:15:23.651875 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-4deb11c6-0b21-42b5-9368-aff6b3be878d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:23.653303 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1af7a0-b951-4328-8b60-ef4a041df598 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:23.656805 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-4deb11c6-0b21-42b5-9368-aff6b3be878d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:23.664869 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 845/1711] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:15:22 2022] DELETE /identity/v3/projects/101bb4a61c984a0498ff7e8bb638bb5f => generated 0 bytes in 948 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:15:23.670841 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e289072f-a6a3-4e6a-8971-907f40cd7d86 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:23.671035 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e289072f-a6a3-4e6a-8971-907f40cd7d86 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:23.673045 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e289072f-a6a3-4e6a-8971-907f40cd7d86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:23.674944 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-bc1af7a0-b951-4328-8b60-ef4a041df598 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:23.844551 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-e289072f-a6a3-4e6a-8971-907f40cd7d86 None None] MFA Rules not processed for user `ad400ce73b644938aef19e2da731c4dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:15:23.866219 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-e289072f-a6a3-4e6a-8971-907f40cd7d86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:24.121402 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1af7a0-b951-4328-8b60-ef4a041df598 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:24.160397 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75df1032-0cef-48f7-9bdf-f6ecb3fc3c3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:24.161535 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75df1032-0cef-48f7-9bdf-f6ecb3fc3c3e admin admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:24.170266 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75df1032-0cef-48f7-9bdf-f6ecb3fc3c3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:24.170978 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75df1032-0cef-48f7-9bdf-f6ecb3fc3c3e admin admin] PATH_INFO: `/v3/users` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:24.172835 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75df1032-0cef-48f7-9bdf-f6ecb3fc3c3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:24.176985 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75df1032-0cef-48f7-9bdf-f6ecb3fc3c3e admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:24.255091 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 846/1712] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:15:23 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 581 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:15:24.255091 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f95105e-74ce-4eb6-9110-ce2d6ef20d3f None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:24.255091 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f95105e-74ce-4eb6-9110-ce2d6ef20d3f None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:24.255091 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f95105e-74ce-4eb6-9110-ce2d6ef20d3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:24.306206 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 867/1713] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:15:23 2022] POST /identity/v3/users => generated 289 bytes in 656 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:15:24.312705 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c854f80-7892-4729-a5c8-2e3acdd730bc None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:24.313355 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c854f80-7892-4729-a5c8-2e3acdd730bc None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:24.313997 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c854f80-7892-4729-a5c8-2e3acdd730bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:24.324489 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-9f95105e-74ce-4eb6-9110-ce2d6ef20d3f None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:15:24.327217 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-9f95105e-74ce-4eb6-9110-ce2d6ef20d3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:24.358665 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 847/1714] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:15:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:15:24.361244 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f95105e-74ce-4eb6-9110-ce2d6ef20d3f None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:24.366571 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-9f95105e-74ce-4eb6-9110-ce2d6ef20d3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:24.431749 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-4c854f80-7892-4729-a5c8-2e3acdd730bc None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:15:24.434181 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-4c854f80-7892-4729-a5c8-2e3acdd730bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:24.487043 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 868/1715] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:15:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:15:24.487043 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c854f80-7892-4729-a5c8-2e3acdd730bc None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:24.495290 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-4c854f80-7892-4729-a5c8-2e3acdd730bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:24.528856 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c854f80-7892-4729-a5c8-2e3acdd730bc None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:24.591657 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c915dade-4117-44f4-b83c-a398371acd7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:24.592349 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c915dade-4117-44f4-b83c-a398371acd7d admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:24.592679 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c915dade-4117-44f4-b83c-a398371acd7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:24.600342 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c915dade-4117-44f4-b83c-a398371acd7d admin admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:24.603652 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c915dade-4117-44f4-b83c-a398371acd7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:24.611640 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c915dade-4117-44f4-b83c-a398371acd7d admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:24.694599 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 869/1716] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:15:24 2022] GET /identity/v3/roles => generated 1420 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:15:24.708779 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c915dade-4117-44f4-b83c-a398371acd7d admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:24.723919 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-c915dade-4117-44f4-b83c-a398371acd7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:24.777990 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f95105e-74ce-4eb6-9110-ce2d6ef20d3f None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:24.817474 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54b25b7-5fb3-4bea-a046-3c5913565312 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5776cbf4f6714b2083c3ebdd6d98cc87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:24.820187 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b25b7-5fb3-4bea-a046-3c5913565312 service nova] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:24.820739 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b25b7-5fb3-4bea-a046-3c5913565312 service nova] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:24.821234 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b25b7-5fb3-4bea-a046-3c5913565312 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:24.856663 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54b25b7-5fb3-4bea-a046-3c5913565312 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:24.861169 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54b25b7-5fb3-4bea-a046-3c5913565312 service nova] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:24.892209 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 848/1717] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:15:24 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 536 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:15:24.907324 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b491cb8e-8adb-412b-8cf5-542a8291ae92 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:24.907826 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b491cb8e-8adb-412b-8cf5-542a8291ae92 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:24.908283 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b491cb8e-8adb-412b-8cf5-542a8291ae92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:25.048529 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-b491cb8e-8adb-412b-8cf5-542a8291ae92 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:15:25.054546 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-b491cb8e-8adb-412b-8cf5-542a8291ae92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:25.091630 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c915dade-4117-44f4-b83c-a398371acd7d admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:25.110388 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb606aa-6a3c-4432-ad1f-c598092f2416 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:25.119814 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb606aa-6a3c-4432-ad1f-c598092f2416 service neutron] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:25.120297 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb606aa-6a3c-4432-ad1f-c598092f2416 service neutron] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:25.122513 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb606aa-6a3c-4432-ad1f-c598092f2416 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:25.126389 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 849/1718] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:15:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:15:25.127351 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b491cb8e-8adb-412b-8cf5-542a8291ae92 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:25.171152 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb606aa-6a3c-4432-ad1f-c598092f2416 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:25.176406 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb606aa-6a3c-4432-ad1f-c598092f2416 service neutron] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:25.181009 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b491cb8e-8adb-412b-8cf5-542a8291ae92 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:25.204113 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333a73b1-2176-4d4e-9de2-ec309bac7260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:25.205284 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333a73b1-2176-4d4e-9de2-ec309bac7260 admin admin] REQUEST_METHOD: `PUT` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:25.205855 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333a73b1-2176-4d4e-9de2-ec309bac7260 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:25.206427 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333a73b1-2176-4d4e-9de2-ec309bac7260 admin admin] PATH_INFO: `/v3/projects/a86c0934a3ed4592ac901ce59849564d/users/cf374d65c1244cb6a3eb59ab6fa4938b/roles/e0ab41d8ae22474688b5361bcc75e0f0` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:25.216054 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:15:25.246387 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 870/1719] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:15:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 556 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:15:25.248568 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb606aa-6a3c-4432-ad1f-c598092f2416 service neutron] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:25.256453 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:15:25.274849 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333a73b1-2176-4d4e-9de2-ec309bac7260 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a86c0934a3ed4592ac901ce59849564d, user_id=cf374d65c1244cb6a3eb59ab6fa4938b, role_id=e0ab41d8ae22474688b5361bcc75e0f0)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:25.292475 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333a73b1-2176-4d4e-9de2-ec309bac7260 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:25.328572 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb606aa-6a3c-4432-ad1f-c598092f2416 service neutron] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:25.339699 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 850/1720] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:15:25 2022] PUT /identity/v3/projects/a86c0934a3ed4592ac901ce59849564d/users/cf374d65c1244cb6a3eb59ab6fa4938b/roles/e0ab41d8ae22474688b5361bcc75e0f0 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:15:25.348769 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333a73b1-2176-4d4e-9de2-ec309bac7260 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:25.633804 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333a73b1-2176-4d4e-9de2-ec309bac7260 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:25.658785 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbf264f-509e-4847-8d60-1500b887aeee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:25.660259 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbf264f-509e-4847-8d60-1500b887aeee admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:25.661003 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbf264f-509e-4847-8d60-1500b887aeee admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:25.661865 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbf264f-509e-4847-8d60-1500b887aeee admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:25.663919 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbf264f-509e-4847-8d60-1500b887aeee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:25.669595 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbf264f-509e-4847-8d60-1500b887aeee admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:25.725835 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 851/1721] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:15:25 2022] GET /identity/v3/roles => generated 1420 bytes in 385 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:15:25.731413 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbf264f-509e-4847-8d60-1500b887aeee admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:25.752239 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbf264f-509e-4847-8d60-1500b887aeee admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:25.759889 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716252c7-18c3-4fe8-b481-ca36a7df99fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:25.761042 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716252c7-18c3-4fe8-b481-ca36a7df99fb admin admin] REQUEST_METHOD: `PUT` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:25.761539 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716252c7-18c3-4fe8-b481-ca36a7df99fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:25.762237 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716252c7-18c3-4fe8-b481-ca36a7df99fb admin admin] PATH_INFO: `/v3/projects/a86c0934a3ed4592ac901ce59849564d/users/cf374d65c1244cb6a3eb59ab6fa4938b/roles/e0ab41d8ae22474688b5361bcc75e0f0` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:25.764917 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716252c7-18c3-4fe8-b481-ca36a7df99fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=a86c0934a3ed4592ac901ce59849564d, user_id=cf374d65c1244cb6a3eb59ab6fa4938b, role_id=e0ab41d8ae22474688b5361bcc75e0f0)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:25.769361 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716252c7-18c3-4fe8-b481-ca36a7df99fb admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:25.786663 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 852/1722] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:15:25 2022] PUT /identity/v3/projects/a86c0934a3ed4592ac901ce59849564d/users/cf374d65c1244cb6a3eb59ab6fa4938b/roles/e0ab41d8ae22474688b5361bcc75e0f0 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:15:25.801545 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f67d61-aac6-42f8-bc21-45e67b268003 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:25.802005 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f67d61-aac6-42f8-bc21-45e67b268003 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:25.802446 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f67d61-aac6-42f8-bc21-45e67b268003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:25.802720 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee402d4-eea6-4a2a-9776-4562cf35a21e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:25.821222 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee402d4-eea6-4a2a-9776-4562cf35a21e admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:25.822050 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee402d4-eea6-4a2a-9776-4562cf35a21e admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:25.822939 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee402d4-eea6-4a2a-9776-4562cf35a21e admin admin] PATH_INFO: `/v3/domains` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:25.831423 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee402d4-eea6-4a2a-9776-4562cf35a21e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:15:25.835358 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee402d4-eea6-4a2a-9776-4562cf35a21e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:25.847249 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee402d4-eea6-4a2a-9776-4562cf35a21e admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:25.873430 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-18f67d61-aac6-42f8-bc21-45e67b268003 None None] MFA Rules not processed for user `cf374d65c1244cb6a3eb59ab6fa4938b`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:15:25.875685 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-18f67d61-aac6-42f8-bc21-45e67b268003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:26.002203 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 871/1723] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 6 20:15:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 755 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:15:26.005069 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a1c17a-05da-403f-b719-0a293f0621e5 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:26.005487 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a1c17a-05da-403f-b719-0a293f0621e5 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:26.006059 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a1c17a-05da-403f-b719-0a293f0621e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:26.201246 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-d0a1c17a-05da-403f-b719-0a293f0621e5 None None] MFA Rules not processed for user `7be957f2c15c4f6f949b3f284a30624d`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:15:26.207256 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-d0a1c17a-05da-403f-b719-0a293f0621e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:26.352841 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 853/1724] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:15:25 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 551 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:15:26.362598 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f67d61-aac6-42f8-bc21-45e67b268003 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:26.836338 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 872/1725] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:15:26 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 833 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:15:26.838762 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9512128f-e221-46e3-9a01-1f5b1d55df3b None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:26.839344 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9512128f-e221-46e3-9a01-1f5b1d55df3b None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:26.839754 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9512128f-e221-46e3-9a01-1f5b1d55df3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:26.880576 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f67d61-aac6-42f8-bc21-45e67b268003 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:26.908942 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b689357-2644-489d-ace1-ab2501779eb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:26.910280 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b689357-2644-489d-ace1-ab2501779eb8 admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:26.911237 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b689357-2644-489d-ace1-ab2501779eb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:26.911733 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b689357-2644-489d-ace1-ab2501779eb8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:26.913553 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b689357-2644-489d-ace1-ab2501779eb8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:26.917645 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b689357-2644-489d-ace1-ab2501779eb8 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:26.960370 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:15:27.012839 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-9512128f-e221-46e3-9a01-1f5b1d55df3b None None] MFA Rules not processed for user `48cca45629ef4e09aac4262616e2792f`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:15:27.015116 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-9512128f-e221-46e3-9a01-1f5b1d55df3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:27.074127 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 854/1726] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:15:26 2022] POST /identity/v3/projects => generated 383 bytes in 720 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:15:27.075899 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b689357-2644-489d-ace1-ab2501779eb8 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:27.082754 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-3b689357-2644-489d-ace1-ab2501779eb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:27.388296 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b689357-2644-489d-ace1-ab2501779eb8 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:27.389911 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 873/1727] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:15:26 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 549 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:15:27.391900 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9512128f-e221-46e3-9a01-1f5b1d55df3b None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:27.410216 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce927a83-b639-488c-89b3-53a579962073 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df1bca993bc142908cc7cf85eabd2b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:27.411336 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce927a83-b639-488c-89b3-53a579962073 service cinder] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:27.419490 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce927a83-b639-488c-89b3-53a579962073 service cinder] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:27.419490 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce927a83-b639-488c-89b3-53a579962073 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:27.463085 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:15:27.799088 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce927a83-b639-488c-89b3-53a579962073 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:27.802705 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9512128f-e221-46e3-9a01-1f5b1d55df3b None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:27.833413 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce927a83-b639-488c-89b3-53a579962073 service cinder] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:27.907343 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 855/1728] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:15:27 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 832 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:15:27.913237 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce927a83-b639-488c-89b3-53a579962073 service cinder] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:28.135920 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aea5b2b-67eb-4721-bd5c-4d90586bd447 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5776cbf4f6714b2083c3ebdd6d98cc87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:28.138965 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aea5b2b-67eb-4721-bd5c-4d90586bd447 service nova] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:28.139365 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aea5b2b-67eb-4721-bd5c-4d90586bd447 service nova] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:28.140527 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aea5b2b-67eb-4721-bd5c-4d90586bd447 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:28.389052 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce927a83-b639-488c-89b3-53a579962073 service cinder] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:28.424205 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2f1e02-da2f-4ead-be3e-26b704057fdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:28.427149 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2f1e02-da2f-4ead-be3e-26b704057fdb admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:28.433900 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2f1e02-da2f-4ead-be3e-26b704057fdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:28.435232 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2f1e02-da2f-4ead-be3e-26b704057fdb admin admin] PATH_INFO: `/v3/users` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:28.436940 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2f1e02-da2f-4ead-be3e-26b704057fdb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:28.440713 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aea5b2b-67eb-4721-bd5c-4d90586bd447 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:28.445031 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2f1e02-da2f-4ead-be3e-26b704057fdb admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:28.447778 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aea5b2b-67eb-4721-bd5c-4d90586bd447 service nova] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:28.620465 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 856/1729] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:15:27 2022] POST /identity/v3/users => generated 292 bytes in 712 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:15:28.624327 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 874/1730] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:15:27 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 1234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:15:28.625074 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2f1e02-da2f-4ead-be3e-26b704057fdb admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:28.635178 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-bd2f1e02-da2f-4ead-be3e-26b704057fdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:28.635826 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073cee7d-ff57-44aa-9022-f2100516bf95 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:28.639610 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073cee7d-ff57-44aa-9022-f2100516bf95 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:28.639610 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073cee7d-ff57-44aa-9022-f2100516bf95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:28.700688 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2f1e02-da2f-4ead-be3e-26b704057fdb admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:28.725235 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eda9077-7244-43d0-a383-1555b27d3597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:28.727664 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eda9077-7244-43d0-a383-1555b27d3597 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:28.728876 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eda9077-7244-43d0-a383-1555b27d3597 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:28.729842 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eda9077-7244-43d0-a383-1555b27d3597 admin admin] PATH_INFO: `/v3/projects/f4638584c5cc46c9b74ef60549cdd7ae` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:28.736812 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eda9077-7244-43d0-a383-1555b27d3597 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4638584c5cc46c9b74ef60549cdd7ae)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:28.757194 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eda9077-7244-43d0-a383-1555b27d3597 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:28.803623 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-073cee7d-ff57-44aa-9022-f2100516bf95 None None] MFA Rules not processed for user `48cca45629ef4e09aac4262616e2792f`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:15:28.831076 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-073cee7d-ff57-44aa-9022-f2100516bf95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:28.847271 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:15:28.984829 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-8eda9077-7244-43d0-a383-1555b27d3597 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4638584c5cc46c9b74ef60549cdd7ae', 'request_id': 'req-8eda9077-7244-43d0-a383-1555b27d3597'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:29.025745 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-8eda9077-7244-43d0-a383-1555b27d3597 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4638584c5cc46c9b74ef60549cdd7ae', 'request_id': 'req-8eda9077-7244-43d0-a383-1555b27d3597'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:29.069666 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-8eda9077-7244-43d0-a383-1555b27d3597 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4638584c5cc46c9b74ef60549cdd7ae', 'request_id': 'req-8eda9077-7244-43d0-a383-1555b27d3597'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:29.096583 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-8eda9077-7244-43d0-a383-1555b27d3597 admin admin] The token cache is being invalidate because project f4638584c5cc46c9b74ef60549cdd7ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81771) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:15:29.096884 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-8eda9077-7244-43d0-a383-1555b27d3597 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:29.097665 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-8eda9077-7244-43d0-a383-1555b27d3597 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:29.101013 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 857/1731] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:15:28 2022] DELETE /identity/v3/projects/f4638584c5cc46c9b74ef60549cdd7ae => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:15:29.105548 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eda9077-7244-43d0-a383-1555b27d3597 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:29.107413 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-8eda9077-7244-43d0-a383-1555b27d3597 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:29.287343 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:15:29.325722 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eda9077-7244-43d0-a383-1555b27d3597 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:29.337892 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397ec383-9550-4691-814a-0e30ac7ce227 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df1bca993bc142908cc7cf85eabd2b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:29.338883 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397ec383-9550-4691-814a-0e30ac7ce227 service cinder] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:29.342107 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397ec383-9550-4691-814a-0e30ac7ce227 service cinder] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:29.342471 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397ec383-9550-4691-814a-0e30ac7ce227 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:29.348622 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-397ec383-9550-4691-814a-0e30ac7ce227 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:29.376226 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 875/1732] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:15:28 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 750 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:15:29.378713 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073cee7d-ff57-44aa-9022-f2100516bf95 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:29.392547 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-073cee7d-ff57-44aa-9022-f2100516bf95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:29.472676 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073cee7d-ff57-44aa-9022-f2100516bf95 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:29.563996 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397ec383-9550-4691-814a-0e30ac7ce227 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:29.565810 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397ec383-9550-4691-814a-0e30ac7ce227 service cinder] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:29.588057 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 858/1733] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:15:29 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:15:29.589168 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397ec383-9550-4691-814a-0e30ac7ce227 service cinder] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:29.590907 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-397ec383-9550-4691-814a-0e30ac7ce227 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:29.866886 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ad24df-f38b-4257-a7c2-038fbb74a293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:29.871145 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ad24df-f38b-4257-a7c2-038fbb74a293 admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:29.871145 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ad24df-f38b-4257-a7c2-038fbb74a293 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:29.871145 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ad24df-f38b-4257-a7c2-038fbb74a293 admin admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:29.871145 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ad24df-f38b-4257-a7c2-038fbb74a293 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:29.879599 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ad24df-f38b-4257-a7c2-038fbb74a293 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:29.914545 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397ec383-9550-4691-814a-0e30ac7ce227 service cinder] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:29.924243 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c7c487-a005-4d34-99bc-f8455defba61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:29.925021 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c7c487-a005-4d34-99bc-f8455defba61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:29.925353 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c7c487-a005-4d34-99bc-f8455defba61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:29.925658 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c7c487-a005-4d34-99bc-f8455defba61 admin admin] PATH_INFO: `/v3/users/bd3e397544154877ab9cd65375189507` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:29.927984 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c7c487-a005-4d34-99bc-f8455defba61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd3e397544154877ab9cd65375189507)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:29.930480 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c7c487-a005-4d34-99bc-f8455defba61 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:29.934134 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:15:29.942595 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 876/1734] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:15:29 2022] GET /identity/v3/roles => generated 1420 bytes in 566 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:15:29.980147 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ad24df-f38b-4257-a7c2-038fbb74a293 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:29.995201 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-07ad24df-f38b-4257-a7c2-038fbb74a293 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:30.109268 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-08c7c487-a005-4d34-99bc-f8455defba61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd3e397544154877ab9cd65375189507'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:30.142880 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-08c7c487-a005-4d34-99bc-f8455defba61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd3e397544154877ab9cd65375189507'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:30.145315 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-08c7c487-a005-4d34-99bc-f8455defba61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd3e397544154877ab9cd65375189507'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:30.146159 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-08c7c487-a005-4d34-99bc-f8455defba61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd3e397544154877ab9cd65375189507'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:30.180981 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-08c7c487-a005-4d34-99bc-f8455defba61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd3e397544154877ab9cd65375189507'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:30.313038 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 859/1735] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:15:29 2022] DELETE /identity/v3/users/bd3e397544154877ab9cd65375189507 => generated 0 bytes in 724 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:15:30.319718 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb466b66-dfa3-4f56-a5c0-aadc270d6198 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:30.322235 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb466b66-dfa3-4f56-a5c0-aadc270d6198 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:30.329001 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb466b66-dfa3-4f56-a5c0-aadc270d6198 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:30.390847 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-eb466b66-dfa3-4f56-a5c0-aadc270d6198 None None] MFA Rules not processed for user `32882d38295046fea004a659a71bde58`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:15:30.396067 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-eb466b66-dfa3-4f56-a5c0-aadc270d6198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:30.400386 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ad24df-f38b-4257-a7c2-038fbb74a293 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:30.549980 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 860/1736] 10.222.0.98 () {58 vars in 1015 bytes} [Tue Sep 6 20:15:30 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:15:30.552148 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb466b66-dfa3-4f56-a5c0-aadc270d6198 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:30.566050 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-eb466b66-dfa3-4f56-a5c0-aadc270d6198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:30.750471 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb466b66-dfa3-4f56-a5c0-aadc270d6198 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:30.764573 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a127207-4a17-4423-9638-067015338e3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:30.766780 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a127207-4a17-4423-9638-067015338e3f admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:30.766780 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a127207-4a17-4423-9638-067015338e3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:30.767432 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a127207-4a17-4423-9638-067015338e3f admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:30.770288 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a127207-4a17-4423-9638-067015338e3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:30.772612 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a127207-4a17-4423-9638-067015338e3f admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:30.800450 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 861/1737] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:15:30 2022] GET /identity/v3/roles => generated 1420 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:15:30.807230 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a127207-4a17-4423-9638-067015338e3f admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:30.816110 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883ce951-c81d-4fa1-a753-1232743cf8e2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5776cbf4f6714b2083c3ebdd6d98cc87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:30.817422 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883ce951-c81d-4fa1-a753-1232743cf8e2 service nova] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:30.821961 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883ce951-c81d-4fa1-a753-1232743cf8e2 service nova] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:30.823583 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883ce951-c81d-4fa1-a753-1232743cf8e2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:30.830425 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-883ce951-c81d-4fa1-a753-1232743cf8e2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:30.834193 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a127207-4a17-4423-9638-067015338e3f admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:30.853166 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8db9ff-af89-4cb7-90fd-6d93c06bd6f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:30.854663 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8db9ff-af89-4cb7-90fd-6d93c06bd6f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:30.855335 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8db9ff-af89-4cb7-90fd-6d93c06bd6f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:30.855946 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8db9ff-af89-4cb7-90fd-6d93c06bd6f5 admin admin] PATH_INFO: `/v3/projects/957fd609924b4f05acfe702c31757437/users/3d9d6f2d326d4695b9f0e727c0bc8aea/roles/4eea39e24b06431aa357053fe7907a3d` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:30.928570 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8db9ff-af89-4cb7-90fd-6d93c06bd6f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=957fd609924b4f05acfe702c31757437, user_id=3d9d6f2d326d4695b9f0e727c0bc8aea, role_id=4eea39e24b06431aa357053fe7907a3d)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:30.944664 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8db9ff-af89-4cb7-90fd-6d93c06bd6f5 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:30.969002 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:15:31.002200 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 862/1738] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:15:30 2022] PUT /identity/v3/projects/957fd609924b4f05acfe702c31757437/users/3d9d6f2d326d4695b9f0e727c0bc8aea/roles/4eea39e24b06431aa357053fe7907a3d => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:15:31.024714 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4fe6a2-f802-49fe-9ee7-2c68c47bb6f9 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:31.025530 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4fe6a2-f802-49fe-9ee7-2c68c47bb6f9 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:31.026305 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4fe6a2-f802-49fe-9ee7-2c68c47bb6f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:31.063460 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:15:31.109266 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883ce951-c81d-4fa1-a753-1232743cf8e2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:31.122080 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883ce951-c81d-4fa1-a753-1232743cf8e2 service nova] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:31.161193 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 877/1739] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:15:29 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 1216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:15:31.257756 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-5b4fe6a2-f802-49fe-9ee7-2c68c47bb6f9 None None] MFA Rules not processed for user `3d9d6f2d326d4695b9f0e727c0bc8aea`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:15:31.269077 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-5b4fe6a2-f802-49fe-9ee7-2c68c47bb6f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:31.288743 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883ce951-c81d-4fa1-a753-1232743cf8e2 service nova] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:31.290690 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-883ce951-c81d-4fa1-a753-1232743cf8e2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:31.327667 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883ce951-c81d-4fa1-a753-1232743cf8e2 service nova] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:31.809007 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dbb6b5-1938-4925-82aa-a83fc85371ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:31.810197 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dbb6b5-1938-4925-82aa-a83fc85371ba service neutron] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:31.810197 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dbb6b5-1938-4925-82aa-a83fc85371ba service neutron] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:31.812161 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dbb6b5-1938-4925-82aa-a83fc85371ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:31.867165 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dbb6b5-1938-4925-82aa-a83fc85371ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:31.873453 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dbb6b5-1938-4925-82aa-a83fc85371ba service neutron] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:31.914002 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 878/1740] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:15:31 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 626 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:15:31.941215 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dbb6b5-1938-4925-82aa-a83fc85371ba service neutron] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:31.941665 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 863/1741] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:15:31 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 919 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:15:32.226411 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dbb6b5-1938-4925-82aa-a83fc85371ba service neutron] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:32.271289 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292f3d8a-fe07-42bb-b9cb-88b47164105f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:32.274329 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292f3d8a-fe07-42bb-b9cb-88b47164105f admin admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:32.274782 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292f3d8a-fe07-42bb-b9cb-88b47164105f admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:32.275369 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292f3d8a-fe07-42bb-b9cb-88b47164105f admin admin] PATH_INFO: `/v3/projects` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:32.288667 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292f3d8a-fe07-42bb-b9cb-88b47164105f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:32.288667 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292f3d8a-fe07-42bb-b9cb-88b47164105f admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:32.388407 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 879/1742] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:15:31 2022] POST /identity/v3/projects => generated 381 bytes in 448 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:15:32.394844 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4fe6a2-f802-49fe-9ee7-2c68c47bb6f9 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:32.432432 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4fe6a2-f802-49fe-9ee7-2c68c47bb6f9 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:32.687729 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799537d1-4a9d-4996-984a-c7d4b3883409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:32.688807 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799537d1-4a9d-4996-984a-c7d4b3883409 admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:32.691376 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799537d1-4a9d-4996-984a-c7d4b3883409 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:32.691836 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799537d1-4a9d-4996-984a-c7d4b3883409 admin admin] PATH_INFO: `/v3/users` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:32.692765 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799537d1-4a9d-4996-984a-c7d4b3883409 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:32.697372 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799537d1-4a9d-4996-984a-c7d4b3883409 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:32.788792 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 864/1743] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:15:32 2022] POST /identity/v3/users => generated 291 bytes in 390 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:15:32.790637 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292f3d8a-fe07-42bb-b9cb-88b47164105f admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:32.822895 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292f3d8a-fe07-42bb-b9cb-88b47164105f admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:32.865303 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075d73d3-8804-40a5-8f2f-b62165bd6e2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:32.872540 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075d73d3-8804-40a5-8f2f-b62165bd6e2c admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:32.873454 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075d73d3-8804-40a5-8f2f-b62165bd6e2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:32.875946 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075d73d3-8804-40a5-8f2f-b62165bd6e2c admin admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:32.883570 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075d73d3-8804-40a5-8f2f-b62165bd6e2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:32.896398 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075d73d3-8804-40a5-8f2f-b62165bd6e2c admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:32.957864 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 880/1744] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:15:32 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:15:32.960697 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799537d1-4a9d-4996-984a-c7d4b3883409 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:33.053858 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799537d1-4a9d-4996-984a-c7d4b3883409 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:33.080673 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b32d63-86a9-4727-ba5a-d8e84046a2f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:33.084426 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b32d63-86a9-4727-ba5a-d8e84046a2f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:33.086663 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b32d63-86a9-4727-ba5a-d8e84046a2f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:33.086982 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b32d63-86a9-4727-ba5a-d8e84046a2f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:33.090780 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b32d63-86a9-4727-ba5a-d8e84046a2f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:33.101822 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b32d63-86a9-4727-ba5a-d8e84046a2f4 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:33.142632 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075d73d3-8804-40a5-8f2f-b62165bd6e2c admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:33.144813 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 865/1745] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:15:32 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:15:33.165781 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075d73d3-8804-40a5-8f2f-b62165bd6e2c admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:33.184048 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4efc450-df6d-4751-8850-022f2c20705f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:33.185128 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4efc450-df6d-4751-8850-022f2c20705f admin admin] REQUEST_METHOD: `PUT` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:33.185460 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4efc450-df6d-4751-8850-022f2c20705f admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:33.185770 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4efc450-df6d-4751-8850-022f2c20705f admin admin] PATH_INFO: `/v3/projects/919bf1f2d52e4526b0a1edcda5a2dd3d/users/66f82d33ee55445fa36af756a2c9f499/roles/4eea39e24b06431aa357053fe7907a3d` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:33.201654 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971f3446-369c-460c-8cd3-116481e5b73e None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:33.201951 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971f3446-369c-460c-8cd3-116481e5b73e None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:33.202256 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971f3446-369c-460c-8cd3-116481e5b73e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:33.260926 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4efc450-df6d-4751-8850-022f2c20705f admin admin] RBAC: Authorizing `identity:create_grant(project_id=919bf1f2d52e4526b0a1edcda5a2dd3d, user_id=66f82d33ee55445fa36af756a2c9f499, role_id=4eea39e24b06431aa357053fe7907a3d)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:33.278169 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4efc450-df6d-4751-8850-022f2c20705f admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:33.286134 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-971f3446-369c-460c-8cd3-116481e5b73e None None] MFA Rules not processed for user `f49ef54a38614cb880577626e65b0e32`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:15:33.291839 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-971f3446-369c-460c-8cd3-116481e5b73e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:33.321295 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 881/1746] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:15:33 2022] PUT /identity/v3/projects/919bf1f2d52e4526b0a1edcda5a2dd3d/users/66f82d33ee55445fa36af756a2c9f499/roles/4eea39e24b06431aa357053fe7907a3d => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 20:15:33.343482 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060e4022-3c36-4d02-ac4a-79feb6888174 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:33.344291 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060e4022-3c36-4d02-ac4a-79feb6888174 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:33.344291 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060e4022-3c36-4d02-ac4a-79feb6888174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:33.415084 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-060e4022-3c36-4d02-ac4a-79feb6888174 None None] MFA Rules not processed for user `66f82d33ee55445fa36af756a2c9f499`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:15:33.416685 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-060e4022-3c36-4d02-ac4a-79feb6888174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:33.438258 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:15:33.644742 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 866/1747] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:15:33 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 440 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:15:33.747000 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 882/1748] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:15:33 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 405 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:15:33.757261 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971f3446-369c-460c-8cd3-116481e5b73e None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:33.841299 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971f3446-369c-460c-8cd3-116481e5b73e None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:33.901487 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:15:34.217576 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7fc37e-7ab8-41b8-a684-396131ea9b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:34.219501 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7fc37e-7ab8-41b8-a684-396131ea9b6d admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:34.221061 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7fc37e-7ab8-41b8-a684-396131ea9b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:34.234181 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7fc37e-7ab8-41b8-a684-396131ea9b6d admin admin] PATH_INFO: `/v3/projects` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:34.238290 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7fc37e-7ab8-41b8-a684-396131ea9b6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:34.251303 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7fc37e-7ab8-41b8-a684-396131ea9b6d admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:34.360865 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 867/1749] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:15:33 2022] POST /identity/v3/projects => generated 385 bytes in 606 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:15:34.367107 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060e4022-3c36-4d02-ac4a-79feb6888174 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:34.392503 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060e4022-3c36-4d02-ac4a-79feb6888174 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:34.602477 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d81ba0-8b9d-406a-a8ef-9a33784c5483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:34.604139 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d81ba0-8b9d-406a-a8ef-9a33784c5483 admin admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:34.604445 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d81ba0-8b9d-406a-a8ef-9a33784c5483 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:34.604722 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d81ba0-8b9d-406a-a8ef-9a33784c5483 admin admin] PATH_INFO: `/v3/users` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:34.605908 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d81ba0-8b9d-406a-a8ef-9a33784c5483 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:34.612531 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d81ba0-8b9d-406a-a8ef-9a33784c5483 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:34.671425 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 883/1750] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:15:34 2022] POST /identity/v3/users => generated 299 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:15:34.676944 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7fc37e-7ab8-41b8-a684-396131ea9b6d admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:34.696035 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7fc37e-7ab8-41b8-a684-396131ea9b6d admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:34.733537 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a77262f-0e52-4c4c-91ad-21822abf3d7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:34.734283 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a77262f-0e52-4c4c-91ad-21822abf3d7c admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:34.734836 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a77262f-0e52-4c4c-91ad-21822abf3d7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:34.735088 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a77262f-0e52-4c4c-91ad-21822abf3d7c admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:34.736197 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a77262f-0e52-4c4c-91ad-21822abf3d7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:34.742039 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a77262f-0e52-4c4c-91ad-21822abf3d7c admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:34.782763 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 868/1751] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:15:34 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:15:34.784156 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d81ba0-8b9d-406a-a8ef-9a33784c5483 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:34.808987 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d81ba0-8b9d-406a-a8ef-9a33784c5483 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:34.836178 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e1d6c8-ec37-4b82-9c27-e154810a4b4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:34.841142 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e1d6c8-ec37-4b82-9c27-e154810a4b4d admin admin] REQUEST_METHOD: `PUT` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:34.841439 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e1d6c8-ec37-4b82-9c27-e154810a4b4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:34.841737 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e1d6c8-ec37-4b82-9c27-e154810a4b4d admin admin] PATH_INFO: `/v3/projects/11622d6408c2496e835711e4e1175e3f/users/162a61788d3d426aa9d5e39e17dd2e90/roles/e0ab41d8ae22474688b5361bcc75e0f0` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:34.954925 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a77262f-0e52-4c4c-91ad-21822abf3d7c admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:34.958084 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-9a77262f-0e52-4c4c-91ad-21822abf3d7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:34.978062 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e1d6c8-ec37-4b82-9c27-e154810a4b4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=11622d6408c2496e835711e4e1175e3f, user_id=162a61788d3d426aa9d5e39e17dd2e90, role_id=e0ab41d8ae22474688b5361bcc75e0f0)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:34.983769 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e1d6c8-ec37-4b82-9c27-e154810a4b4d admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:35.049123 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 884/1752] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:15:34 2022] PUT /identity/v3/projects/11622d6408c2496e835711e4e1175e3f/users/162a61788d3d426aa9d5e39e17dd2e90/roles/e0ab41d8ae22474688b5361bcc75e0f0 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:15:35.054657 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e1d6c8-ec37-4b82-9c27-e154810a4b4d admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:35.063421 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a77262f-0e52-4c4c-91ad-21822abf3d7c admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:35.357971 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c637fd3-dadb-421a-8128-7a7e6bc4d23c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:35.359657 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c637fd3-dadb-421a-8128-7a7e6bc4d23c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:35.361995 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c637fd3-dadb-421a-8128-7a7e6bc4d23c admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:35.362808 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c637fd3-dadb-421a-8128-7a7e6bc4d23c admin admin] PATH_INFO: `/v3/users/50f1911c4a5743aeb1c1ee91c524c9b7` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:35.365405 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c637fd3-dadb-421a-8128-7a7e6bc4d23c admin admin] RBAC: Authorizing `identity:delete_user(user_id=50f1911c4a5743aeb1c1ee91c524c9b7)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:35.375711 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c637fd3-dadb-421a-8128-7a7e6bc4d23c admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:35.418144 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e1d6c8-ec37-4b82-9c27-e154810a4b4d admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:35.432851 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb19c378-87ce-4dac-9949-259289b70d98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:35.433807 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb19c378-87ce-4dac-9949-259289b70d98 admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:35.434206 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb19c378-87ce-4dac-9949-259289b70d98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:35.434622 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb19c378-87ce-4dac-9949-259289b70d98 admin admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:35.435513 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb19c378-87ce-4dac-9949-259289b70d98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:35.439303 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb19c378-87ce-4dac-9949-259289b70d98 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:35.477628 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 885/1753] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:15:35 2022] GET /identity/v3/roles => generated 1420 bytes in 419 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:15:35.477628 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb19c378-87ce-4dac-9949-259289b70d98 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:35.478783 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-fb19c378-87ce-4dac-9949-259289b70d98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:35.522697 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb19c378-87ce-4dac-9949-259289b70d98 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:35.567771 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53c1997-3d51-4b10-9e27-495f2b994bf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:35.576114 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53c1997-3d51-4b10-9e27-495f2b994bf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:35.576483 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53c1997-3d51-4b10-9e27-495f2b994bf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:35.576762 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53c1997-3d51-4b10-9e27-495f2b994bf8 admin admin] PATH_INFO: `/v3/projects/54061e8ce554448989fe72a89b243f9b` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:35.586599 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53c1997-3d51-4b10-9e27-495f2b994bf8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=54061e8ce554448989fe72a89b243f9b)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:35.590274 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53c1997-3d51-4b10-9e27-495f2b994bf8 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:35.609972 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:15:35.675002 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-7c637fd3-dadb-421a-8128-7a7e6bc4d23c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50f1911c4a5743aeb1c1ee91c524c9b7'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:35.703302 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-7c637fd3-dadb-421a-8128-7a7e6bc4d23c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50f1911c4a5743aeb1c1ee91c524c9b7'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:35.707105 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-7c637fd3-dadb-421a-8128-7a7e6bc4d23c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50f1911c4a5743aeb1c1ee91c524c9b7'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:35.713815 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-7c637fd3-dadb-421a-8128-7a7e6bc4d23c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50f1911c4a5743aeb1c1ee91c524c9b7'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:35.780395 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-7c637fd3-dadb-421a-8128-7a7e6bc4d23c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50f1911c4a5743aeb1c1ee91c524c9b7'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:35.949981 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 869/1754] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:15:34 2022] DELETE /identity/v3/users/50f1911c4a5743aeb1c1ee91c524c9b7 => generated 0 bytes in 995 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 20:15:35.952590 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c637fd3-dadb-421a-8128-7a7e6bc4d23c admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:35.966855 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-7c637fd3-dadb-421a-8128-7a7e6bc4d23c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:35.999296 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-a53c1997-3d51-4b10-9e27-495f2b994bf8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54061e8ce554448989fe72a89b243f9b', 'request_id': 'req-a53c1997-3d51-4b10-9e27-495f2b994bf8'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:36.043095 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-a53c1997-3d51-4b10-9e27-495f2b994bf8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54061e8ce554448989fe72a89b243f9b', 'request_id': 'req-a53c1997-3d51-4b10-9e27-495f2b994bf8'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:36.051474 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:15:36.103121 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-a53c1997-3d51-4b10-9e27-495f2b994bf8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54061e8ce554448989fe72a89b243f9b', 'request_id': 'req-a53c1997-3d51-4b10-9e27-495f2b994bf8'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:36.168050 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-a53c1997-3d51-4b10-9e27-495f2b994bf8 admin admin] The token cache is being invalidate because project 54061e8ce554448989fe72a89b243f9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81772) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:15:36.168357 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-a53c1997-3d51-4b10-9e27-495f2b994bf8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:36.170192 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-a53c1997-3d51-4b10-9e27-495f2b994bf8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:36.175404 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 886/1755] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:15:35 2022] DELETE /identity/v3/projects/54061e8ce554448989fe72a89b243f9b => generated 0 bytes in 701 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:15:36.181785 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472c974f-665c-48b2-8920-94f939738f8f None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:36.182093 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472c974f-665c-48b2-8920-94f939738f8f None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:36.182415 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472c974f-665c-48b2-8920-94f939738f8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:36.272783 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-472c974f-665c-48b2-8920-94f939738f8f None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:15:36.278517 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-472c974f-665c-48b2-8920-94f939738f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:36.493123 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c637fd3-dadb-421a-8128-7a7e6bc4d23c admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:36.549220 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c58725c-8c79-4265-82b7-9952a141a642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:36.551487 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c58725c-8c79-4265-82b7-9952a141a642 admin admin] REQUEST_METHOD: `PUT` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:36.554108 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c58725c-8c79-4265-82b7-9952a141a642 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:36.562262 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c58725c-8c79-4265-82b7-9952a141a642 admin admin] PATH_INFO: `/v3/projects/11622d6408c2496e835711e4e1175e3f/users/162a61788d3d426aa9d5e39e17dd2e90/roles/e0ab41d8ae22474688b5361bcc75e0f0` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:36.572559 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c58725c-8c79-4265-82b7-9952a141a642 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11622d6408c2496e835711e4e1175e3f, user_id=162a61788d3d426aa9d5e39e17dd2e90, role_id=e0ab41d8ae22474688b5361bcc75e0f0)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:36.584745 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c58725c-8c79-4265-82b7-9952a141a642 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:36.634881 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 870/1756] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:15:35 2022] PUT /identity/v3/projects/11622d6408c2496e835711e4e1175e3f/users/162a61788d3d426aa9d5e39e17dd2e90/roles/e0ab41d8ae22474688b5361bcc75e0f0 => generated 0 bytes in 685 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:15:36.651997 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2c0989-c8e3-4352-a4ee-d9758b991287 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:36.651997 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2c0989-c8e3-4352-a4ee-d9758b991287 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:36.652457 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 887/1757] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:15:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 466 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:15:36.653311 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2c0989-c8e3-4352-a4ee-d9758b991287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:36.653793 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469a7959-f9e7-4e4f-b37a-8b32408d42e3 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:36.653966 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469a7959-f9e7-4e4f-b37a-8b32408d42e3 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:36.654221 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469a7959-f9e7-4e4f-b37a-8b32408d42e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:36.708142 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-7e2c0989-c8e3-4352-a4ee-d9758b991287 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:15:36.712749 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-7e2c0989-c8e3-4352-a4ee-d9758b991287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:36.742495 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-469a7959-f9e7-4e4f-b37a-8b32408d42e3 None None] MFA Rules not processed for user `162a61788d3d426aa9d5e39e17dd2e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:15:36.743784 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-469a7959-f9e7-4e4f-b37a-8b32408d42e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:36.964785 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 871/1758] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:15:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:15:36.978379 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b454c1a-fd0a-40d7-bfb5-b3b93a25c86a None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:36.978735 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b454c1a-fd0a-40d7-bfb5-b3b93a25c86a None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:36.978876 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b454c1a-fd0a-40d7-bfb5-b3b93a25c86a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:37.074473 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 888/1759] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:15:36 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 421 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:15:37.078918 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-1b454c1a-fd0a-40d7-bfb5-b3b93a25c86a None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:15:37.086022 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-1b454c1a-fd0a-40d7-bfb5-b3b93a25c86a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:37.103789 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469a7959-f9e7-4e4f-b37a-8b32408d42e3 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:37.107034 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-469a7959-f9e7-4e4f-b37a-8b32408d42e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:37.133852 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 872/1760] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:15:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:15:37.136856 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b454c1a-fd0a-40d7-bfb5-b3b93a25c86a None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:37.182715 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:15:37.199476 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b454c1a-fd0a-40d7-bfb5-b3b93a25c86a None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:37.224027 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd951e6-c32f-478b-bb28-da04ca8e300b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:37.224942 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd951e6-c32f-478b-bb28-da04ca8e300b admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:37.225414 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd951e6-c32f-478b-bb28-da04ca8e300b admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:37.225889 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd951e6-c32f-478b-bb28-da04ca8e300b admin admin] PATH_INFO: `/v3/domains` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:37.227072 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd951e6-c32f-478b-bb28-da04ca8e300b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:15:37.227596 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd951e6-c32f-478b-bb28-da04ca8e300b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:37.262798 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd951e6-c32f-478b-bb28-da04ca8e300b admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:37.324618 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 873/1761] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 6 20:15:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:15:37.325659 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd951e6-c32f-478b-bb28-da04ca8e300b admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:37.344914 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-0fd951e6-c32f-478b-bb28-da04ca8e300b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:37.367273 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469a7959-f9e7-4e4f-b37a-8b32408d42e3 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:37.384122 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a76825-0453-47ab-853c-dbed4ac713b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df1bca993bc142908cc7cf85eabd2b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:37.386643 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a76825-0453-47ab-853c-dbed4ac713b3 service cinder] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:37.390283 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a76825-0453-47ab-853c-dbed4ac713b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:37.390734 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a76825-0453-47ab-853c-dbed4ac713b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:37.401685 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-14a76825-0453-47ab-853c-dbed4ac713b3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:37.634213 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd951e6-c32f-478b-bb28-da04ca8e300b admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:37.663845 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0709180-b3b1-468c-bf55-2f02b3ec7691 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:37.664494 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0709180-b3b1-468c-bf55-2f02b3ec7691 service neutron] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:37.664777 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0709180-b3b1-468c-bf55-2f02b3ec7691 service neutron] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:37.670019 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0709180-b3b1-468c-bf55-2f02b3ec7691 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:37.704960 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0709180-b3b1-468c-bf55-2f02b3ec7691 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:37.707028 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0709180-b3b1-468c-bf55-2f02b3ec7691 service neutron] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:37.734635 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a76825-0453-47ab-853c-dbed4ac713b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:37.736317 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a76825-0453-47ab-853c-dbed4ac713b3 service cinder] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:37.763911 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 874/1762] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:15:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:15:37.766308 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a59eea8-759a-49af-a0aa-a9ce1d249df5 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:37.766664 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a59eea8-759a-49af-a0aa-a9ce1d249df5 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:37.766980 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a59eea8-759a-49af-a0aa-a9ce1d249df5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:37.820538 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 889/1763] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:15:37 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 717 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 20:15:37.821783 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a76825-0453-47ab-853c-dbed4ac713b3 service cinder] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:37.822949 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-6a59eea8-759a-49af-a0aa-a9ce1d249df5 None None] MFA Rules not processed for user `32882d38295046fea004a659a71bde58`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:15:37.827810 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-6a59eea8-759a-49af-a0aa-a9ce1d249df5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:37.880228 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 875/1764] 10.222.0.20 () {58 vars in 1009 bytes} [Tue Sep 6 20:15:37 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:15:37.889369 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a76825-0453-47ab-853c-dbed4ac713b3 service cinder] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:37.919814 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ced1686-848f-4b5b-b4a8-494400bb49f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:37.922794 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ced1686-848f-4b5b-b4a8-494400bb49f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:37.923213 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ced1686-848f-4b5b-b4a8-494400bb49f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:37.923633 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ced1686-848f-4b5b-b4a8-494400bb49f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:37.928340 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ced1686-848f-4b5b-b4a8-494400bb49f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:37.940654 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ced1686-848f-4b5b-b4a8-494400bb49f8 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:38.115387 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 890/1765] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:15:37 2022] POST /identity/v3/projects => generated 385 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:15:38.116839 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a59eea8-759a-49af-a0aa-a9ce1d249df5 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:38.160732 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a59eea8-759a-49af-a0aa-a9ce1d249df5 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:38.231691 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:15:38.288114 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ced1686-848f-4b5b-b4a8-494400bb49f8 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:38.291702 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-1ced1686-848f-4b5b-b4a8-494400bb49f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:38.379784 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f0f508-f42d-4caf-9f72-42a7a29edebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:38.381555 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f0f508-f42d-4caf-9f72-42a7a29edebf admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:38.384290 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f0f508-f42d-4caf-9f72-42a7a29edebf admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:38.384705 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f0f508-f42d-4caf-9f72-42a7a29edebf admin admin] PATH_INFO: `/v3/users` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:38.386412 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f0f508-f42d-4caf-9f72-42a7a29edebf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:38.392956 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f0f508-f42d-4caf-9f72-42a7a29edebf admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:38.461535 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 876/1766] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:15:38 2022] POST /identity/v3/users => generated 293 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:15:38.468812 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f0f508-f42d-4caf-9f72-42a7a29edebf admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:38.543745 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f0f508-f42d-4caf-9f72-42a7a29edebf admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:38.571395 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ced1686-848f-4b5b-b4a8-494400bb49f8 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:38.575800 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0388fa70-0aa2-477a-9df2-55986e628bbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:38.576749 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0388fa70-0aa2-477a-9df2-55986e628bbf admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:38.577041 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0388fa70-0aa2-477a-9df2-55986e628bbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:38.581763 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0388fa70-0aa2-477a-9df2-55986e628bbf admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:38.596687 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0388fa70-0aa2-477a-9df2-55986e628bbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:38.600847 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0388fa70-0aa2-477a-9df2-55986e628bbf admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:38.601822 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec6804c-40de-473b-85c2-5fe1bff3a26d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:38.603707 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec6804c-40de-473b-85c2-5fe1bff3a26d service neutron] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:38.603985 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec6804c-40de-473b-85c2-5fe1bff3a26d service neutron] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:38.606483 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec6804c-40de-473b-85c2-5fe1bff3a26d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:38.644784 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec6804c-40de-473b-85c2-5fe1bff3a26d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:38.652590 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 877/1767] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:15:38 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:15:38.653437 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec6804c-40de-473b-85c2-5fe1bff3a26d service neutron] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:38.654240 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0388fa70-0aa2-477a-9df2-55986e628bbf admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:38.699266 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 891/1768] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:15:38 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:15:38.702253 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0388fa70-0aa2-477a-9df2-55986e628bbf admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:38.710207 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e492ec55-7454-4737-b2fd-8e27cb2093b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:38.711161 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e492ec55-7454-4737-b2fd-8e27cb2093b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:38.711456 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e492ec55-7454-4737-b2fd-8e27cb2093b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:38.711734 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e492ec55-7454-4737-b2fd-8e27cb2093b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:38.712719 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e492ec55-7454-4737-b2fd-8e27cb2093b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:38.714684 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e492ec55-7454-4737-b2fd-8e27cb2093b7 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:38.737248 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 878/1769] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:15:38 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:15:38.739880 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec6804c-40de-473b-85c2-5fe1bff3a26d service neutron] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:38.772139 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec6804c-40de-473b-85c2-5fe1bff3a26d service neutron] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:38.779245 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2336c7-a1a6-4404-97a9-4a271bfa56c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:38.779968 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2336c7-a1a6-4404-97a9-4a271bfa56c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:38.780255 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2336c7-a1a6-4404-97a9-4a271bfa56c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:38.780682 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2336c7-a1a6-4404-97a9-4a271bfa56c3 admin admin] PATH_INFO: `/v3/projects/f0b532ee73074f40a5935bc70a6989cf/users/fc803fc7a2ac4d09aec68ad6b0630aff/roles/4eea39e24b06431aa357053fe7907a3d` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:38.848019 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2336c7-a1a6-4404-97a9-4a271bfa56c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0b532ee73074f40a5935bc70a6989cf, user_id=fc803fc7a2ac4d09aec68ad6b0630aff, role_id=4eea39e24b06431aa357053fe7907a3d)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:38.852697 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2336c7-a1a6-4404-97a9-4a271bfa56c3 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:38.917682 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 892/1770] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:15:38 2022] PUT /identity/v3/projects/f0b532ee73074f40a5935bc70a6989cf/users/fc803fc7a2ac4d09aec68ad6b0630aff/roles/4eea39e24b06431aa357053fe7907a3d => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:15:38.928492 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818ddc4e-864a-4c77-80fb-ad0869b5768d None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:38.928807 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818ddc4e-864a-4c77-80fb-ad0869b5768d None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:38.929243 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818ddc4e-864a-4c77-80fb-ad0869b5768d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:39.017249 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-818ddc4e-864a-4c77-80fb-ad0869b5768d None None] MFA Rules not processed for user `fc803fc7a2ac4d09aec68ad6b0630aff`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:15:39.018760 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-818ddc4e-864a-4c77-80fb-ad0869b5768d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:39.337283 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 879/1771] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:15:38 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 410 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Sep 06 20:15:39.342237 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2336c7-a1a6-4404-97a9-4a271bfa56c3 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:39.436736 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:15:39.586104 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2336c7-a1a6-4404-97a9-4a271bfa56c3 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:39.612639 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7d5e47-73b8-4882-9709-331a30bce543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:39.621751 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7d5e47-73b8-4882-9709-331a30bce543 admin admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:39.622359 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7d5e47-73b8-4882-9709-331a30bce543 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:39.622846 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7d5e47-73b8-4882-9709-331a30bce543 admin admin] PATH_INFO: `/v3/projects` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:39.627224 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7d5e47-73b8-4882-9709-331a30bce543 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:39.652117 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7d5e47-73b8-4882-9709-331a30bce543 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:39.889336 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 893/1772] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:15:39 2022] POST /identity/v3/projects => generated 387 bytes in 544 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 20:15:39.891387 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818ddc4e-864a-4c77-80fb-ad0869b5768d None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:39.928089 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818ddc4e-864a-4c77-80fb-ad0869b5768d None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:40.200734 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d113efa9-8f51-43cf-8df3-b80a3395b926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:40.202513 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d113efa9-8f51-43cf-8df3-b80a3395b926 admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:40.203001 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d113efa9-8f51-43cf-8df3-b80a3395b926 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:40.203539 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d113efa9-8f51-43cf-8df3-b80a3395b926 admin admin] PATH_INFO: `/v3/users` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:40.210749 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d113efa9-8f51-43cf-8df3-b80a3395b926 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:40.215850 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d113efa9-8f51-43cf-8df3-b80a3395b926 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:40.335941 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 880/1773] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:15:39 2022] POST /identity/v3/users => generated 300 bytes in 441 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:15:40.367250 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7d5e47-73b8-4882-9709-331a30bce543 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:40.432247 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7d5e47-73b8-4882-9709-331a30bce543 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:40.464344 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690a89bb-1d33-4a8c-b189-3a1996aa98b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:40.466102 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690a89bb-1d33-4a8c-b189-3a1996aa98b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:40.466834 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690a89bb-1d33-4a8c-b189-3a1996aa98b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:40.467546 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690a89bb-1d33-4a8c-b189-3a1996aa98b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:40.474211 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690a89bb-1d33-4a8c-b189-3a1996aa98b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:40.480359 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690a89bb-1d33-4a8c-b189-3a1996aa98b7 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:40.542234 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 894/1774] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:15:40 2022] GET /identity/v3/roles => generated 1420 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:15:40.548004 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d113efa9-8f51-43cf-8df3-b80a3395b926 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:40.568227 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d113efa9-8f51-43cf-8df3-b80a3395b926 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:40.583652 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce9cc6d-a86f-4238-9f3b-126f8b5dc754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:40.585058 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce9cc6d-a86f-4238-9f3b-126f8b5dc754 admin admin] REQUEST_METHOD: `PUT` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:40.585502 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce9cc6d-a86f-4238-9f3b-126f8b5dc754 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:40.586005 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce9cc6d-a86f-4238-9f3b-126f8b5dc754 admin admin] PATH_INFO: `/v3/projects/fc5ac4e95d36421f9f0e27e4f381f1ad/users/b78f579fc6544baf85b70dbbb8b73873/roles/e0ab41d8ae22474688b5361bcc75e0f0` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:40.590867 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:15:40.654633 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce9cc6d-a86f-4238-9f3b-126f8b5dc754 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc5ac4e95d36421f9f0e27e4f381f1ad, user_id=b78f579fc6544baf85b70dbbb8b73873, role_id=e0ab41d8ae22474688b5361bcc75e0f0)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:40.659857 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce9cc6d-a86f-4238-9f3b-126f8b5dc754 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:40.722620 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 881/1775] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:15:40 2022] PUT /identity/v3/projects/fc5ac4e95d36421f9f0e27e4f381f1ad/users/b78f579fc6544baf85b70dbbb8b73873/roles/e0ab41d8ae22474688b5361bcc75e0f0 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:15:40.724879 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690a89bb-1d33-4a8c-b189-3a1996aa98b7 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:40.805879 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690a89bb-1d33-4a8c-b189-3a1996aa98b7 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:40.986407 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce9cc6d-a86f-4238-9f3b-126f8b5dc754 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:40.990220 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-6ce9cc6d-a86f-4238-9f3b-126f8b5dc754 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:41.047460 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a018b55e-77ab-4459-a4c3-c62d1bfa786d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:41.049981 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a018b55e-77ab-4459-a4c3-c62d1bfa786d admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:41.050499 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a018b55e-77ab-4459-a4c3-c62d1bfa786d admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:41.054744 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a018b55e-77ab-4459-a4c3-c62d1bfa786d admin admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:41.057412 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a018b55e-77ab-4459-a4c3-c62d1bfa786d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:41.064129 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a018b55e-77ab-4459-a4c3-c62d1bfa786d admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:41.081674 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 895/1776] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:15:40 2022] GET /identity/v3/roles => generated 1420 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:15:41.095625 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a018b55e-77ab-4459-a4c3-c62d1bfa786d admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:41.112247 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a018b55e-77ab-4459-a4c3-c62d1bfa786d admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:41.119150 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9301b8c1-8d96-4a3c-8e9c-422ef11f8059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:41.120654 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9301b8c1-8d96-4a3c-8e9c-422ef11f8059 admin admin] REQUEST_METHOD: `PUT` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:41.121326 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9301b8c1-8d96-4a3c-8e9c-422ef11f8059 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:41.121915 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9301b8c1-8d96-4a3c-8e9c-422ef11f8059 admin admin] PATH_INFO: `/v3/projects/fc5ac4e95d36421f9f0e27e4f381f1ad/users/b78f579fc6544baf85b70dbbb8b73873/roles/e0ab41d8ae22474688b5361bcc75e0f0` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:41.131875 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9301b8c1-8d96-4a3c-8e9c-422ef11f8059 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc5ac4e95d36421f9f0e27e4f381f1ad, user_id=b78f579fc6544baf85b70dbbb8b73873, role_id=e0ab41d8ae22474688b5361bcc75e0f0)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:41.139860 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9301b8c1-8d96-4a3c-8e9c-422ef11f8059 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:41.158924 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 896/1777] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:15:41 2022] PUT /identity/v3/projects/fc5ac4e95d36421f9f0e27e4f381f1ad/users/b78f579fc6544baf85b70dbbb8b73873/roles/e0ab41d8ae22474688b5361bcc75e0f0 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:15:41.184554 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bd4d8e-8fe1-450d-9068-438f24ad4410 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:41.186898 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bd4d8e-8fe1-450d-9068-438f24ad4410 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:41.188668 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bd4d8e-8fe1-450d-9068-438f24ad4410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:41.254416 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce9cc6d-a86f-4238-9f3b-126f8b5dc754 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:41.287927 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-e1bd4d8e-8fe1-450d-9068-438f24ad4410 None None] MFA Rules not processed for user `b78f579fc6544baf85b70dbbb8b73873`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:15:41.290182 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-e1bd4d8e-8fe1-450d-9068-438f24ad4410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:41.507029 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b47992-982c-48d1-b63f-0b6e3c4f4761 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df1bca993bc142908cc7cf85eabd2b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:41.507976 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b47992-982c-48d1-b63f-0b6e3c4f4761 service cinder] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:41.508485 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b47992-982c-48d1-b63f-0b6e3c4f4761 service cinder] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:41.509075 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b47992-982c-48d1-b63f-0b6e3c4f4761 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:41.519052 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-a4b47992-982c-48d1-b63f-0b6e3c4f4761 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:41.554276 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 897/1778] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:15:41 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 373 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:15:41.573324 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bd4d8e-8fe1-450d-9068-438f24ad4410 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:41.599083 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bd4d8e-8fe1-450d-9068-438f24ad4410 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:41.604255 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74830347-b360-4889-b46a-5a5115eb4e7a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df1bca993bc142908cc7cf85eabd2b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:41.605015 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74830347-b360-4889-b46a-5a5115eb4e7a service cinder] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:41.605300 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74830347-b360-4889-b46a-5a5115eb4e7a service cinder] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:41.607760 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74830347-b360-4889-b46a-5a5115eb4e7a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:41.627137 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74830347-b360-4889-b46a-5a5115eb4e7a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:41.628688 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74830347-b360-4889-b46a-5a5115eb4e7a service cinder] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:41.644356 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:15:41.653842 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 898/1779] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:15:41 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 20:15:41.797226 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b47992-982c-48d1-b63f-0b6e3c4f4761 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:41.799082 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b47992-982c-48d1-b63f-0b6e3c4f4761 service cinder] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:41.831880 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 882/1780] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:15:40 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 845 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 20:15:41.842560 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413d662e-f9a6-4415-b626-42f3e8cd7a7d None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:41.843511 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413d662e-f9a6-4415-b626-42f3e8cd7a7d None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:41.844017 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413d662e-f9a6-4415-b626-42f3e8cd7a7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:41.884456 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b47992-982c-48d1-b63f-0b6e3c4f4761 service cinder] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:41.984818 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-413d662e-f9a6-4415-b626-42f3e8cd7a7d None None] MFA Rules not processed for user `ad400ce73b644938aef19e2da731c4dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:15:41.986235 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-413d662e-f9a6-4415-b626-42f3e8cd7a7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:42.077670 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:15:42.136017 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b47992-982c-48d1-b63f-0b6e3c4f4761 service cinder] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:42.177431 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3913add-0a9c-4a64-8838-f8416a410c90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:42.178705 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3913add-0a9c-4a64-8838-f8416a410c90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:42.179282 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3913add-0a9c-4a64-8838-f8416a410c90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:42.179777 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3913add-0a9c-4a64-8838-f8416a410c90 admin admin] PATH_INFO: `/v3/users/fc803fc7a2ac4d09aec68ad6b0630aff` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:42.190728 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3913add-0a9c-4a64-8838-f8416a410c90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc803fc7a2ac4d09aec68ad6b0630aff)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:42.194028 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3913add-0a9c-4a64-8838-f8416a410c90 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:42.240127 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 899/1781] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:15:41 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 400 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:15:42.511880 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-c3913add-0a9c-4a64-8838-f8416a410c90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc803fc7a2ac4d09aec68ad6b0630aff'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:42.552904 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-c3913add-0a9c-4a64-8838-f8416a410c90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc803fc7a2ac4d09aec68ad6b0630aff'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:42.555971 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-c3913add-0a9c-4a64-8838-f8416a410c90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc803fc7a2ac4d09aec68ad6b0630aff'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:42.557228 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-c3913add-0a9c-4a64-8838-f8416a410c90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc803fc7a2ac4d09aec68ad6b0630aff'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:42.576335 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-c3913add-0a9c-4a64-8838-f8416a410c90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc803fc7a2ac4d09aec68ad6b0630aff'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:42.628324 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 883/1782] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:15:41 2022] DELETE /identity/v3/users/fc803fc7a2ac4d09aec68ad6b0630aff => generated 0 bytes in 744 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:15:42.635099 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d754586-621a-4429-baf3-612b865c6e08 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:42.635595 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d754586-621a-4429-baf3-612b865c6e08 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:42.636095 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d754586-621a-4429-baf3-612b865c6e08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:42.751366 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-6d754586-621a-4429-baf3-612b865c6e08 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:15:42.761316 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-6d754586-621a-4429-baf3-612b865c6e08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:43.101054 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 900/1783] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:15:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 468 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:15:43.197696 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3913add-0a9c-4a64-8838-f8416a410c90 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:43.200092 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-c3913add-0a9c-4a64-8838-f8416a410c90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:43.386688 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0824e57e-527c-49a0-a76d-1a4d26a36f64 None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:43.388595 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0824e57e-527c-49a0-a76d-1a4d26a36f64 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:43.389677 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0824e57e-527c-49a0-a76d-1a4d26a36f64 None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:43.395866 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 901/1784] 10.222.0.20 () {54 vars in 921 bytes} [Tue Sep 6 20:15:43 2022] GET /identity => generated 270 bytes in 24 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:15:43.424615 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3913add-0a9c-4a64-8838-f8416a410c90 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:43.446029 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2488c72f-90a9-4051-af66-2c8fed3f303c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:43.447946 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2488c72f-90a9-4051-af66-2c8fed3f303c service neutron] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:43.449761 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2488c72f-90a9-4051-af66-2c8fed3f303c service neutron] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:43.450649 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2488c72f-90a9-4051-af66-2c8fed3f303c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:43.484520 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2488c72f-90a9-4051-af66-2c8fed3f303c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:43.489520 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2488c72f-90a9-4051-af66-2c8fed3f303c service neutron] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:43.543423 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 884/1785] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:15:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:15:43.604511 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5222bb-f246-400f-a91f-4f902f3f4de2 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:43.606808 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5222bb-f246-400f-a91f-4f902f3f4de2 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:43.608511 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5222bb-f246-400f-a91f-4f902f3f4de2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:43.725493 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-ec5222bb-f246-400f-a91f-4f902f3f4de2 None None] MFA Rules not processed for user `5776cbf4f6714b2083c3ebdd6d98cc87`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:15:43.726811 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-ec5222bb-f246-400f-a91f-4f902f3f4de2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:43.946039 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2488c72f-90a9-4051-af66-2c8fed3f303c service neutron] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:43.950292 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-2488c72f-90a9-4051-af66-2c8fed3f303c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:44.127419 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 902/1786] 10.222.0.20 () {58 vars in 1018 bytes} [Tue Sep 6 20:15:43 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 531 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:15:44.250280 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2488c72f-90a9-4051-af66-2c8fed3f303c service neutron] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:44.266517 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851a41e7-f15c-437a-b9ba-04edd36d76a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df1bca993bc142908cc7cf85eabd2b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:44.267132 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851a41e7-f15c-437a-b9ba-04edd36d76a0 service cinder] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:44.267421 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851a41e7-f15c-437a-b9ba-04edd36d76a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:44.270866 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1978da37-de40-4f9d-be3f-470a04080ab7 None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:44.271258 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1978da37-de40-4f9d-be3f-470a04080ab7 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:44.272313 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1978da37-de40-4f9d-be3f-470a04080ab7 None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:44.274226 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 903/1787] 10.222.0.20 () {54 vars in 921 bytes} [Tue Sep 6 20:15:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 20:15:44.274689 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851a41e7-f15c-437a-b9ba-04edd36d76a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:44.280103 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-851a41e7-f15c-437a-b9ba-04edd36d76a0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:44.360016 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eb015f-af6f-4879-bef2-710c794389c0 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:44.360308 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eb015f-af6f-4879-bef2-710c794389c0 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:44.360586 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eb015f-af6f-4879-bef2-710c794389c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:44.377431 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:15:44.416090 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-22eb015f-af6f-4879-bef2-710c794389c0 None None] MFA Rules not processed for user `5776cbf4f6714b2083c3ebdd6d98cc87`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:15:44.423032 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-22eb015f-af6f-4879-bef2-710c794389c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:44.455795 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 904/1788] 10.222.0.20 () {58 vars in 1018 bytes} [Tue Sep 6 20:15:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:15:44.475256 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851a41e7-f15c-437a-b9ba-04edd36d76a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:44.477964 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851a41e7-f15c-437a-b9ba-04edd36d76a0 service cinder] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:44.492989 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 885/1789] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:15:43 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 550 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:15:44.609313 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eb015f-af6f-4879-bef2-710c794389c0 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:44.611036 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-22eb015f-af6f-4879-bef2-710c794389c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:44.684621 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eb015f-af6f-4879-bef2-710c794389c0 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:44.699646 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7d549d-962a-415d-97ba-bc8cc19212f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:44.700315 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7d549d-962a-415d-97ba-bc8cc19212f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:44.700589 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7d549d-962a-415d-97ba-bc8cc19212f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:44.702862 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7d549d-962a-415d-97ba-bc8cc19212f9 admin admin] PATH_INFO: `/v3/projects/44654764d56041cda9b28cd3f6953729` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:44.710544 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7d549d-962a-415d-97ba-bc8cc19212f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=44654764d56041cda9b28cd3f6953729)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:44.721159 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7d549d-962a-415d-97ba-bc8cc19212f9 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:44.814166 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:15:44.876895 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-ed7d549d-962a-415d-97ba-bc8cc19212f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44654764d56041cda9b28cd3f6953729', 'request_id': 'req-ed7d549d-962a-415d-97ba-bc8cc19212f9'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:44.897983 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-ed7d549d-962a-415d-97ba-bc8cc19212f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44654764d56041cda9b28cd3f6953729', 'request_id': 'req-ed7d549d-962a-415d-97ba-bc8cc19212f9'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:44.917682 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-ed7d549d-962a-415d-97ba-bc8cc19212f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44654764d56041cda9b28cd3f6953729', 'request_id': 'req-ed7d549d-962a-415d-97ba-bc8cc19212f9'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:44.985086 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-ed7d549d-962a-415d-97ba-bc8cc19212f9 admin admin] The token cache is being invalidate because project 44654764d56041cda9b28cd3f6953729 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81772) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:15:44.985086 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-ed7d549d-962a-415d-97ba-bc8cc19212f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:44.987385 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-ed7d549d-962a-415d-97ba-bc8cc19212f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:44.998067 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 905/1790] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:15:44 2022] DELETE /identity/v3/projects/44654764d56041cda9b28cd3f6953729 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:15:45.007561 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851a41e7-f15c-437a-b9ba-04edd36d76a0 service cinder] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:45.062792 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851a41e7-f15c-437a-b9ba-04edd36d76a0 service cinder] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:45.261006 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006789a8-b610-43e4-b75e-a7ff0186e1a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:45.262022 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006789a8-b610-43e4-b75e-a7ff0186e1a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:45.262155 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006789a8-b610-43e4-b75e-a7ff0186e1a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:45.262318 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006789a8-b610-43e4-b75e-a7ff0186e1a2 admin admin] PATH_INFO: `/v3/users/cf374d65c1244cb6a3eb59ab6fa4938b` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:45.269840 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006789a8-b610-43e4-b75e-a7ff0186e1a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf374d65c1244cb6a3eb59ab6fa4938b)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:45.272909 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006789a8-b610-43e4-b75e-a7ff0186e1a2 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:45.418893 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-006789a8-b610-43e4-b75e-a7ff0186e1a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf374d65c1244cb6a3eb59ab6fa4938b'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:45.439748 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-006789a8-b610-43e4-b75e-a7ff0186e1a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf374d65c1244cb6a3eb59ab6fa4938b'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:45.440808 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-006789a8-b610-43e4-b75e-a7ff0186e1a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf374d65c1244cb6a3eb59ab6fa4938b'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:45.441853 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-006789a8-b610-43e4-b75e-a7ff0186e1a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf374d65c1244cb6a3eb59ab6fa4938b'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:45.466008 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-006789a8-b610-43e4-b75e-a7ff0186e1a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf374d65c1244cb6a3eb59ab6fa4938b'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:45.502898 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 886/1791] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:15:45 2022] DELETE /identity/v3/users/cf374d65c1244cb6a3eb59ab6fa4938b => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:15:46.340886 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7d549d-962a-415d-97ba-bc8cc19212f9 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:46.343352 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-ed7d549d-962a-415d-97ba-bc8cc19212f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:46.522303 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7d549d-962a-415d-97ba-bc8cc19212f9 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:46.595624 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e629972-11f3-45c8-b492-8449aa0361bc None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:46.596689 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e629972-11f3-45c8-b492-8449aa0361bc None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:46.597914 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e629972-11f3-45c8-b492-8449aa0361bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:46.642773 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8bf1ea-2f78-4c96-9d6f-e25c1337cbd7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5776cbf4f6714b2083c3ebdd6d98cc87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:46.643491 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8bf1ea-2f78-4c96-9d6f-e25c1337cbd7 service nova] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:46.643807 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8bf1ea-2f78-4c96-9d6f-e25c1337cbd7 service nova] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:46.644086 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8bf1ea-2f78-4c96-9d6f-e25c1337cbd7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:46.646565 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-4f8bf1ea-2f78-4c96-9d6f-e25c1337cbd7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:46.664879 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8bf1ea-2f78-4c96-9d6f-e25c1337cbd7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:46.667932 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8bf1ea-2f78-4c96-9d6f-e25c1337cbd7 service nova] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:46.680907 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-3e629972-11f3-45c8-b492-8449aa0361bc None None] MFA Rules not processed for user `7be957f2c15c4f6f949b3f284a30624d`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:15:46.682448 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-3e629972-11f3-45c8-b492-8449aa0361bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:46.685234 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 906/1792] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:15:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:15:46.687455 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8bf1ea-2f78-4c96-9d6f-e25c1337cbd7 service nova] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:46.689102 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-4f8bf1ea-2f78-4c96-9d6f-e25c1337cbd7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:46.789355 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:15:46.803225 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:15:46.826618 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 887/1793] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:15:46 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:15:46.861457 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8bf1ea-2f78-4c96-9d6f-e25c1337cbd7 service nova] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:46.877864 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a735cbe5-e8a3-404a-8c64-06092b14c7d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:46.878943 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a735cbe5-e8a3-404a-8c64-06092b14c7d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:46.880714 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a735cbe5-e8a3-404a-8c64-06092b14c7d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:46.881725 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a735cbe5-e8a3-404a-8c64-06092b14c7d9 admin admin] PATH_INFO: `/v3/projects/f0b532ee73074f40a5935bc70a6989cf` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:46.883289 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a735cbe5-e8a3-404a-8c64-06092b14c7d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0b532ee73074f40a5935bc70a6989cf)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:46.887577 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a735cbe5-e8a3-404a-8c64-06092b14c7d9 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:47.041881 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e629972-11f3-45c8-b492-8449aa0361bc None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:47.044234 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-3e629972-11f3-45c8-b492-8449aa0361bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:47.082188 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e629972-11f3-45c8-b492-8449aa0361bc None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:47.093025 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-a735cbe5-e8a3-404a-8c64-06092b14c7d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0b532ee73074f40a5935bc70a6989cf', 'request_id': 'req-a735cbe5-e8a3-404a-8c64-06092b14c7d9'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:47.113080 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-a735cbe5-e8a3-404a-8c64-06092b14c7d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0b532ee73074f40a5935bc70a6989cf', 'request_id': 'req-a735cbe5-e8a3-404a-8c64-06092b14c7d9'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:47.127224 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-a735cbe5-e8a3-404a-8c64-06092b14c7d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0b532ee73074f40a5935bc70a6989cf', 'request_id': 'req-a735cbe5-e8a3-404a-8c64-06092b14c7d9'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:47.179091 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-a735cbe5-e8a3-404a-8c64-06092b14c7d9 admin admin] The token cache is being invalidate because project f0b532ee73074f40a5935bc70a6989cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81772) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:15:47.179410 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-a735cbe5-e8a3-404a-8c64-06092b14c7d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:47.182415 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-a735cbe5-e8a3-404a-8c64-06092b14c7d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:47.194186 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 907/1794] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:15:46 2022] DELETE /identity/v3/projects/f0b532ee73074f40a5935bc70a6989cf => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:15:47.196355 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a735cbe5-e8a3-404a-8c64-06092b14c7d9 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:47.200932 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-a735cbe5-e8a3-404a-8c64-06092b14c7d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:47.277554 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8f4c6d-3795-4da6-a28c-452d5195f52b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:47.278988 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8f4c6d-3795-4da6-a28c-452d5195f52b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:47.279522 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8f4c6d-3795-4da6-a28c-452d5195f52b admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:47.279988 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8f4c6d-3795-4da6-a28c-452d5195f52b admin admin] PATH_INFO: `/v3/projects/a86c0934a3ed4592ac901ce59849564d` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:47.297087 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8f4c6d-3795-4da6-a28c-452d5195f52b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a86c0934a3ed4592ac901ce59849564d)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:47.302358 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8f4c6d-3795-4da6-a28c-452d5195f52b admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:47.402438 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a735cbe5-e8a3-404a-8c64-06092b14c7d9 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:47.418356 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5ab555-d1e6-4e8f-b176-4d62401b9ddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:47.419839 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5ab555-d1e6-4e8f-b176-4d62401b9ddd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:47.423612 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5ab555-d1e6-4e8f-b176-4d62401b9ddd admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:47.424130 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5ab555-d1e6-4e8f-b176-4d62401b9ddd admin admin] PATH_INFO: `/v3/users/b78f579fc6544baf85b70dbbb8b73873` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:47.427266 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5ab555-d1e6-4e8f-b176-4d62401b9ddd admin admin] RBAC: Authorizing `identity:delete_user(user_id=b78f579fc6544baf85b70dbbb8b73873)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:47.430066 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5ab555-d1e6-4e8f-b176-4d62401b9ddd admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:47.468492 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-6e8f4c6d-3795-4da6-a28c-452d5195f52b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a86c0934a3ed4592ac901ce59849564d', 'request_id': 'req-6e8f4c6d-3795-4da6-a28c-452d5195f52b'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:47.491651 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-6e8f4c6d-3795-4da6-a28c-452d5195f52b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a86c0934a3ed4592ac901ce59849564d', 'request_id': 'req-6e8f4c6d-3795-4da6-a28c-452d5195f52b'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:47.517161 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-6e8f4c6d-3795-4da6-a28c-452d5195f52b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a86c0934a3ed4592ac901ce59849564d', 'request_id': 'req-6e8f4c6d-3795-4da6-a28c-452d5195f52b'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:47.546555 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-6e8f4c6d-3795-4da6-a28c-452d5195f52b admin admin] The token cache is being invalidate because project a86c0934a3ed4592ac901ce59849564d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81771) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:15:47.546870 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-6e8f4c6d-3795-4da6-a28c-452d5195f52b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:47.548559 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-5d5ab555-d1e6-4e8f-b176-4d62401b9ddd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b78f579fc6544baf85b70dbbb8b73873'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:47.549409 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-6e8f4c6d-3795-4da6-a28c-452d5195f52b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:47.550150 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-5d5ab555-d1e6-4e8f-b176-4d62401b9ddd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b78f579fc6544baf85b70dbbb8b73873'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:47.554329 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 888/1795] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:15:47 2022] DELETE /identity/v3/projects/a86c0934a3ed4592ac901ce59849564d => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:15:47.562999 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8f4c6d-3795-4da6-a28c-452d5195f52b admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:47.567487 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-6e8f4c6d-3795-4da6-a28c-452d5195f52b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:47.600005 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-5d5ab555-d1e6-4e8f-b176-4d62401b9ddd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b78f579fc6544baf85b70dbbb8b73873'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:47.601519 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-5d5ab555-d1e6-4e8f-b176-4d62401b9ddd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b78f579fc6544baf85b70dbbb8b73873'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:47.618757 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:15:47.634653 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-5d5ab555-d1e6-4e8f-b176-4d62401b9ddd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b78f579fc6544baf85b70dbbb8b73873'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:47.676705 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 908/1796] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:15:47 2022] DELETE /identity/v3/users/b78f579fc6544baf85b70dbbb8b73873 => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:15:47.679077 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e84d9-566b-4be0-9718-dc10bc3389f5 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:47.679720 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e84d9-566b-4be0-9718-dc10bc3389f5 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:47.680496 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e84d9-566b-4be0-9718-dc10bc3389f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:47.708520 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-7c5e84d9-566b-4be0-9718-dc10bc3389f5 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:15:47.710437 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-7c5e84d9-566b-4be0-9718-dc10bc3389f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:47.788178 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8f4c6d-3795-4da6-a28c-452d5195f52b admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:47.842230 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:15:47.903971 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 909/1797] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:15:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:15:47.907452 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e034b70-222e-43d8-9433-a4aa4d725474 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:47.907735 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e034b70-222e-43d8-9433-a4aa4d725474 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:47.908043 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e034b70-222e-43d8-9433-a4aa4d725474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:47.993197 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-0e034b70-222e-43d8-9433-a4aa4d725474 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:15:47.994584 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-0e034b70-222e-43d8-9433-a4aa4d725474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:48.019636 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 910/1798] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:15:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:15:48.025056 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e034b70-222e-43d8-9433-a4aa4d725474 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:48.040279 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec418e4-ea94-4b94-b5d1-c6b07c1c93da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5776cbf4f6714b2083c3ebdd6d98cc87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:48.042771 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec418e4-ea94-4b94-b5d1-c6b07c1c93da service nova] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:48.043164 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec418e4-ea94-4b94-b5d1-c6b07c1c93da service nova] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:48.043642 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec418e4-ea94-4b94-b5d1-c6b07c1c93da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:48.045486 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e034b70-222e-43d8-9433-a4aa4d725474 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:48.061854 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7338af5e-f30c-439e-a22e-6a56c48631e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:48.062439 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-2ec418e4-ea94-4b94-b5d1-c6b07c1c93da service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:48.062615 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7338af5e-f30c-439e-a22e-6a56c48631e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:48.062900 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7338af5e-f30c-439e-a22e-6a56c48631e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:48.063169 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7338af5e-f30c-439e-a22e-6a56c48631e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:48.063844 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7338af5e-f30c-439e-a22e-6a56c48631e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:15:48.064192 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7338af5e-f30c-439e-a22e-6a56c48631e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:48.066022 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7338af5e-f30c-439e-a22e-6a56c48631e3 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:48.091922 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec418e4-ea94-4b94-b5d1-c6b07c1c93da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:48.094618 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec418e4-ea94-4b94-b5d1-c6b07c1c93da service nova] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:48.102274 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 911/1799] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 6 20:15:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:15:48.107757 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7338af5e-f30c-439e-a22e-6a56c48631e3 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:48.115438 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 889/1800] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:15:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 560 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:15:48.130405 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7338af5e-f30c-439e-a22e-6a56c48631e3 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:48.143318 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cb0082-d9ef-4cf8-9922-642e99721765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:48.143791 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cb0082-d9ef-4cf8-9922-642e99721765 admin admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:48.144209 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cb0082-d9ef-4cf8-9922-642e99721765 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:48.144661 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cb0082-d9ef-4cf8-9922-642e99721765 admin admin] PATH_INFO: `/v3/projects` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:48.145532 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cb0082-d9ef-4cf8-9922-642e99721765 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:48.148320 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cb0082-d9ef-4cf8-9922-642e99721765 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:48.244895 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 912/1801] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:15:48 2022] POST /identity/v3/projects => generated 377 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 20:15:48.250696 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec418e4-ea94-4b94-b5d1-c6b07c1c93da service nova] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:48.266842 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec418e4-ea94-4b94-b5d1-c6b07c1c93da service nova] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:48.453045 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8010df39-7678-45fe-95a0-1a51fcc94dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:48.454539 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8010df39-7678-45fe-95a0-1a51fcc94dc1 admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:48.455240 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8010df39-7678-45fe-95a0-1a51fcc94dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:48.456186 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8010df39-7678-45fe-95a0-1a51fcc94dc1 admin admin] PATH_INFO: `/v3/users` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:48.458856 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8010df39-7678-45fe-95a0-1a51fcc94dc1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:48.461374 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8010df39-7678-45fe-95a0-1a51fcc94dc1 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:48.518679 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 890/1802] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:15:48 2022] POST /identity/v3/users => generated 289 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:15:48.521324 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cb0082-d9ef-4cf8-9922-642e99721765 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:48.542967 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8010df39-7678-45fe-95a0-1a51fcc94dc1 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:48.550833 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-8010df39-7678-45fe-95a0-1a51fcc94dc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:48.567509 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cb0082-d9ef-4cf8-9922-642e99721765 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:48.587345 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac86ad38-bdb6-4944-9753-8ffc631a6dcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:48.587975 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac86ad38-bdb6-4944-9753-8ffc631a6dcf admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:48.588410 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac86ad38-bdb6-4944-9753-8ffc631a6dcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:48.588767 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac86ad38-bdb6-4944-9753-8ffc631a6dcf admin admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:48.592673 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac86ad38-bdb6-4944-9753-8ffc631a6dcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:48.595164 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac86ad38-bdb6-4944-9753-8ffc631a6dcf admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:48.606323 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8010df39-7678-45fe-95a0-1a51fcc94dc1 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:48.622371 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e5f137-8006-4877-a901-a8dcc8e7d35d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:48.623435 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e5f137-8006-4877-a901-a8dcc8e7d35d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:48.624500 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e5f137-8006-4877-a901-a8dcc8e7d35d admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:48.624736 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e5f137-8006-4877-a901-a8dcc8e7d35d admin admin] PATH_INFO: `/v3/projects/fc5ac4e95d36421f9f0e27e4f381f1ad` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:48.631880 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e5f137-8006-4877-a901-a8dcc8e7d35d admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc5ac4e95d36421f9f0e27e4f381f1ad)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:48.636429 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 913/1803] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:15:48 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:15:48.637201 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e5f137-8006-4877-a901-a8dcc8e7d35d admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:48.639023 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac86ad38-bdb6-4944-9753-8ffc631a6dcf admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:48.704208 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac86ad38-bdb6-4944-9753-8ffc631a6dcf admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:48.770708 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fe1e7a-c6a3-488a-b46b-f7cba4d41f01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:48.772827 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fe1e7a-c6a3-488a-b46b-f7cba4d41f01 admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:48.773150 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fe1e7a-c6a3-488a-b46b-f7cba4d41f01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:48.773712 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fe1e7a-c6a3-488a-b46b-f7cba4d41f01 admin admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:48.774514 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fe1e7a-c6a3-488a-b46b-f7cba4d41f01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:48.776357 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fe1e7a-c6a3-488a-b46b-f7cba4d41f01 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:48.803012 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 914/1804] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:15:48 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 20:15:48.812982 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fe1e7a-c6a3-488a-b46b-f7cba4d41f01 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:48.860122 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fe1e7a-c6a3-488a-b46b-f7cba4d41f01 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:48.875972 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-d5e5f137-8006-4877-a901-a8dcc8e7d35d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc5ac4e95d36421f9f0e27e4f381f1ad', 'request_id': 'req-d5e5f137-8006-4877-a901-a8dcc8e7d35d'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:48.889464 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-d5e5f137-8006-4877-a901-a8dcc8e7d35d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc5ac4e95d36421f9f0e27e4f381f1ad', 'request_id': 'req-d5e5f137-8006-4877-a901-a8dcc8e7d35d'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:48.906325 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-d5e5f137-8006-4877-a901-a8dcc8e7d35d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc5ac4e95d36421f9f0e27e4f381f1ad', 'request_id': 'req-d5e5f137-8006-4877-a901-a8dcc8e7d35d'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:48.959891 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-d5e5f137-8006-4877-a901-a8dcc8e7d35d admin admin] The token cache is being invalidate because project fc5ac4e95d36421f9f0e27e4f381f1ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81771) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:15:48.963921 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-d5e5f137-8006-4877-a901-a8dcc8e7d35d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:48.967791 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-d5e5f137-8006-4877-a901-a8dcc8e7d35d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:48.970729 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 891/1805] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:15:48 2022] DELETE /identity/v3/projects/fc5ac4e95d36421f9f0e27e4f381f1ad => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:15:48.977919 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd46700e-85eb-4eb7-b3d0-d15bf9d5ff02 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:48.978454 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd46700e-85eb-4eb7-b3d0-d15bf9d5ff02 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:48.978930 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd46700e-85eb-4eb7-b3d0-d15bf9d5ff02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:49.036437 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755fdc03-97ad-49f7-9020-7ddab3e86357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:49.042513 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755fdc03-97ad-49f7-9020-7ddab3e86357 admin admin] REQUEST_METHOD: `PUT` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:49.043125 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755fdc03-97ad-49f7-9020-7ddab3e86357 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:49.043599 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755fdc03-97ad-49f7-9020-7ddab3e86357 admin admin] PATH_INFO: `/v3/projects/95cb7f1253d644cba5e3309588410579/users/390fc4ffdaf24a7d887c10e4a7a8928b/roles/4eea39e24b06431aa357053fe7907a3d` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:49.066428 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-dd46700e-85eb-4eb7-b3d0-d15bf9d5ff02 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:15:49.076125 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-dd46700e-85eb-4eb7-b3d0-d15bf9d5ff02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:49.109168 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 892/1806] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:15:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:15:49.112407 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755fdc03-97ad-49f7-9020-7ddab3e86357 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95cb7f1253d644cba5e3309588410579, user_id=390fc4ffdaf24a7d887c10e4a7a8928b, role_id=4eea39e24b06431aa357053fe7907a3d)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:49.121122 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755fdc03-97ad-49f7-9020-7ddab3e86357 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:49.125917 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd35a87-a475-4694-a02a-9358cb9b74bf None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:49.126390 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd35a87-a475-4694-a02a-9358cb9b74bf None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:49.129177 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd35a87-a475-4694-a02a-9358cb9b74bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:49.182811 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 915/1807] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:15:48 2022] PUT /identity/v3/projects/95cb7f1253d644cba5e3309588410579/users/390fc4ffdaf24a7d887c10e4a7a8928b/roles/4eea39e24b06431aa357053fe7907a3d => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:15:49.187173 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ad98be-613b-4f96-ac92-521ee8b5c9a2 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:49.187522 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ad98be-613b-4f96-ac92-521ee8b5c9a2 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:49.188672 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ad98be-613b-4f96-ac92-521ee8b5c9a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:49.198904 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:DELETE=1,keystone:SELECT=97,keystone:INSERT=2 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:15:49.232796 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-30ad98be-613b-4f96-ac92-521ee8b5c9a2 None None] MFA Rules not processed for user `390fc4ffdaf24a7d887c10e4a7a8928b`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:15:49.235141 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-3dd35a87-a475-4694-a02a-9358cb9b74bf None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:15:49.237752 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-30ad98be-613b-4f96-ac92-521ee8b5c9a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:49.242814 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-3dd35a87-a475-4694-a02a-9358cb9b74bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:49.254380 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:15:49.404459 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 916/1808] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:15:49 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:15:49.405950 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ad98be-613b-4f96-ac92-521ee8b5c9a2 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:49.415873 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-30ad98be-613b-4f96-ac92-521ee8b5c9a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:49.434264 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 893/1809] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:15:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:15:49.435866 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd35a87-a475-4694-a02a-9358cb9b74bf None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:49.437946 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-3dd35a87-a475-4694-a02a-9358cb9b74bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:49.459468 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd35a87-a475-4694-a02a-9358cb9b74bf None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:49.466042 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe5ad4c-0b00-4f69-9e01-f5b970d8eb86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:49.466879 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe5ad4c-0b00-4f69-9e01-f5b970d8eb86 admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:49.467184 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe5ad4c-0b00-4f69-9e01-f5b970d8eb86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:49.467528 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe5ad4c-0b00-4f69-9e01-f5b970d8eb86 admin admin] PATH_INFO: `/v3/projects` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:49.468207 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe5ad4c-0b00-4f69-9e01-f5b970d8eb86 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:49.470654 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe5ad4c-0b00-4f69-9e01-f5b970d8eb86 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:49.531912 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 894/1810] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:15:49 2022] POST /identity/v3/projects => generated 375 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:15:49.531912 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe5ad4c-0b00-4f69-9e01-f5b970d8eb86 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:49.606771 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ad98be-613b-4f96-ac92-521ee8b5c9a2 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:49.730144 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe5ad4c-0b00-4f69-9e01-f5b970d8eb86 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:49.740761 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9dd0fb-fe36-482c-aa83-51fb514f0d02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:49.742288 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9dd0fb-fe36-482c-aa83-51fb514f0d02 admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:49.742794 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9dd0fb-fe36-482c-aa83-51fb514f0d02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:49.743440 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9dd0fb-fe36-482c-aa83-51fb514f0d02 admin admin] PATH_INFO: `/v3/domains` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:49.744582 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9dd0fb-fe36-482c-aa83-51fb514f0d02 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:15:49.745095 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9dd0fb-fe36-482c-aa83-51fb514f0d02 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:49.748452 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9dd0fb-fe36-482c-aa83-51fb514f0d02 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:49.781722 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc2ed8e-5155-4a8b-8ed5-ec832fa451e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:49.783564 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 895/1811] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 6 20:15:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:15:49.785115 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9dd0fb-fe36-482c-aa83-51fb514f0d02 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:49.785537 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc2ed8e-5155-4a8b-8ed5-ec832fa451e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:49.787710 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc2ed8e-5155-4a8b-8ed5-ec832fa451e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:49.788625 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc2ed8e-5155-4a8b-8ed5-ec832fa451e0 admin admin] PATH_INFO: `/v3/users/3d9d6f2d326d4695b9f0e727c0bc8aea` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:49.791613 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc2ed8e-5155-4a8b-8ed5-ec832fa451e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d9d6f2d326d4695b9f0e727c0bc8aea)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:49.799201 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc2ed8e-5155-4a8b-8ed5-ec832fa451e0 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:49.804854 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9dd0fb-fe36-482c-aa83-51fb514f0d02 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:49.815399 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7863f86-c2f9-4b89-a2cf-e4ed9127603b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:49.816697 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7863f86-c2f9-4b89-a2cf-e4ed9127603b admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:49.817229 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7863f86-c2f9-4b89-a2cf-e4ed9127603b admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:49.817887 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7863f86-c2f9-4b89-a2cf-e4ed9127603b admin admin] PATH_INFO: `/v3/users` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:49.820428 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7863f86-c2f9-4b89-a2cf-e4ed9127603b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:49.824323 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7863f86-c2f9-4b89-a2cf-e4ed9127603b admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:49.880392 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 896/1812] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:15:49 2022] POST /identity/v3/users => generated 294 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:15:49.882100 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7863f86-c2f9-4b89-a2cf-e4ed9127603b admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:49.904896 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7863f86-c2f9-4b89-a2cf-e4ed9127603b admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:49.912618 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e498911-42f1-48bf-a4bf-f75aed410836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:49.913400 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e498911-42f1-48bf-a4bf-f75aed410836 admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:49.913728 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e498911-42f1-48bf-a4bf-f75aed410836 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:49.914031 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e498911-42f1-48bf-a4bf-f75aed410836 admin admin] PATH_INFO: `/v3/projects` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:49.914914 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e498911-42f1-48bf-a4bf-f75aed410836 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:49.918285 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e498911-42f1-48bf-a4bf-f75aed410836 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:49.981551 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 897/1813] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:15:49 2022] POST /identity/v3/projects => generated 385 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:15:49.993242 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e498911-42f1-48bf-a4bf-f75aed410836 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:50.065408 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:15:50.078567 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-0fc2ed8e-5155-4a8b-8ed5-ec832fa451e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d9d6f2d326d4695b9f0e727c0bc8aea'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:50.081030 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-0fc2ed8e-5155-4a8b-8ed5-ec832fa451e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d9d6f2d326d4695b9f0e727c0bc8aea'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:50.136667 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-0fc2ed8e-5155-4a8b-8ed5-ec832fa451e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d9d6f2d326d4695b9f0e727c0bc8aea'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:50.142149 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:15:50.142971 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-0fc2ed8e-5155-4a8b-8ed5-ec832fa451e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d9d6f2d326d4695b9f0e727c0bc8aea'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:50.174421 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-0fc2ed8e-5155-4a8b-8ed5-ec832fa451e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d9d6f2d326d4695b9f0e727c0bc8aea'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:50.188907 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e498911-42f1-48bf-a4bf-f75aed410836 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:50.196971 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d611308-c07f-4a6e-9875-12cc52989849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:50.199127 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d611308-c07f-4a6e-9875-12cc52989849 admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:50.199518 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d611308-c07f-4a6e-9875-12cc52989849 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:50.199751 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d611308-c07f-4a6e-9875-12cc52989849 admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:50.200410 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d611308-c07f-4a6e-9875-12cc52989849 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:50.204667 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d611308-c07f-4a6e-9875-12cc52989849 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:50.231754 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 898/1814] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:15:49 2022] GET /identity/v3/roles => generated 1420 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:15:50.231754 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d611308-c07f-4a6e-9875-12cc52989849 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:50.236144 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-6d611308-c07f-4a6e-9875-12cc52989849 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:50.252779 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 917/1815] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:15:49 2022] DELETE /identity/v3/users/3d9d6f2d326d4695b9f0e727c0bc8aea => generated 0 bytes in 847 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:15:50.259772 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d611308-c07f-4a6e-9875-12cc52989849 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:50.262026 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc2ed8e-5155-4a8b-8ed5-ec832fa451e0 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:50.266748 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-0fc2ed8e-5155-4a8b-8ed5-ec832fa451e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:50.420733 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3d1c0f-579b-4566-b459-67277ceaca34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:50.421915 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3d1c0f-579b-4566-b459-67277ceaca34 admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:50.422258 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3d1c0f-579b-4566-b459-67277ceaca34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:50.435205 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3d1c0f-579b-4566-b459-67277ceaca34 admin admin] PATH_INFO: `/v3/users` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:50.437464 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3d1c0f-579b-4566-b459-67277ceaca34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:50.443846 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3d1c0f-579b-4566-b459-67277ceaca34 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:50.509785 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 899/1816] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:15:50 2022] POST /identity/v3/users => generated 293 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:15:50.512761 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a6906d-6899-4772-b6d0-0c927ecd25f8 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:50.513543 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a6906d-6899-4772-b6d0-0c927ecd25f8 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:50.514286 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a6906d-6899-4772-b6d0-0c927ecd25f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:50.575500 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-66a6906d-6899-4772-b6d0-0c927ecd25f8 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:15:50.581383 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-66a6906d-6899-4772-b6d0-0c927ecd25f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:50.624670 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 900/1817] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:15:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:15:50.626351 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a6906d-6899-4772-b6d0-0c927ecd25f8 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:50.636767 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc2ed8e-5155-4a8b-8ed5-ec832fa451e0 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:50.664306 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f828561-9f15-4bb5-bd1c-d5803ae4cf42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:50.676775 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a6906d-6899-4772-b6d0-0c927ecd25f8 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:50.678035 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f828561-9f15-4bb5-bd1c-d5803ae4cf42 admin admin] REQUEST_METHOD: `PUT` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:50.678543 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f828561-9f15-4bb5-bd1c-d5803ae4cf42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:50.679051 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f828561-9f15-4bb5-bd1c-d5803ae4cf42 admin admin] PATH_INFO: `/v3/projects/0912369993c943529c87b2f8d6e02d1e/users/1f61a795f30a420c83aa65cebab57c60/roles/e0ab41d8ae22474688b5361bcc75e0f0` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:50.695589 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06143f15-21c7-4880-bab8-005656398137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:50.696909 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06143f15-21c7-4880-bab8-005656398137 admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:50.697653 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06143f15-21c7-4880-bab8-005656398137 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:50.698306 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06143f15-21c7-4880-bab8-005656398137 admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:50.701963 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06143f15-21c7-4880-bab8-005656398137 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:50.708258 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06143f15-21c7-4880-bab8-005656398137 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:50.740999 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 901/1818] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:15:50 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:15:50.747382 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f828561-9f15-4bb5-bd1c-d5803ae4cf42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0912369993c943529c87b2f8d6e02d1e, user_id=1f61a795f30a420c83aa65cebab57c60, role_id=e0ab41d8ae22474688b5361bcc75e0f0)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:50.751796 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06143f15-21c7-4880-bab8-005656398137 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:50.760493 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f828561-9f15-4bb5-bd1c-d5803ae4cf42 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:50.775155 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06143f15-21c7-4880-bab8-005656398137 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:50.783890 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db23b7f3-fa74-4f24-a8f6-7d7d11a23181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:50.785155 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db23b7f3-fa74-4f24-a8f6-7d7d11a23181 admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:50.785722 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db23b7f3-fa74-4f24-a8f6-7d7d11a23181 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:50.786169 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db23b7f3-fa74-4f24-a8f6-7d7d11a23181 admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:50.788420 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db23b7f3-fa74-4f24-a8f6-7d7d11a23181 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:50.791219 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db23b7f3-fa74-4f24-a8f6-7d7d11a23181 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:50.810723 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 918/1819] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:15:50 2022] PUT /identity/v3/projects/0912369993c943529c87b2f8d6e02d1e/users/1f61a795f30a420c83aa65cebab57c60/roles/e0ab41d8ae22474688b5361bcc75e0f0 => generated 0 bytes in 558 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:15:50.817313 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f828561-9f15-4bb5-bd1c-d5803ae4cf42 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:50.828228 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 902/1820] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:15:50 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:15:50.834985 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db23b7f3-fa74-4f24-a8f6-7d7d11a23181 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:50.870905 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db23b7f3-fa74-4f24-a8f6-7d7d11a23181 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:51.033651 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f828561-9f15-4bb5-bd1c-d5803ae4cf42 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:51.043805 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67653862-3a6e-4eea-8dc8-3ac03e7f9bb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:51.044751 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67653862-3a6e-4eea-8dc8-3ac03e7f9bb8 admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:51.045218 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67653862-3a6e-4eea-8dc8-3ac03e7f9bb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:51.045686 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67653862-3a6e-4eea-8dc8-3ac03e7f9bb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:51.047721 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67653862-3a6e-4eea-8dc8-3ac03e7f9bb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:51.052698 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67653862-3a6e-4eea-8dc8-3ac03e7f9bb8 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:51.080953 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 919/1821] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:15:50 2022] GET /identity/v3/roles => generated 1420 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 20:15:51.087714 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67653862-3a6e-4eea-8dc8-3ac03e7f9bb8 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:51.134585 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67653862-3a6e-4eea-8dc8-3ac03e7f9bb8 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:51.143521 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb5b667-c2d5-4072-9b8e-011ac7a18001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:51.149335 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb5b667-c2d5-4072-9b8e-011ac7a18001 admin admin] REQUEST_METHOD: `PUT` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:51.150675 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb5b667-c2d5-4072-9b8e-011ac7a18001 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:51.152416 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e099a01c-83eb-421e-a2f6-79e3fe63f042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:51.152632 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb5b667-c2d5-4072-9b8e-011ac7a18001 admin admin] PATH_INFO: `/v3/projects/982ddc3647c446de8a81f3dbce35e35f/users/13c49de541c749779489b9e8314736cf/roles/4eea39e24b06431aa357053fe7907a3d` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:51.155651 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e099a01c-83eb-421e-a2f6-79e3fe63f042 admin admin] REQUEST_METHOD: `PUT` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:51.156346 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e099a01c-83eb-421e-a2f6-79e3fe63f042 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:51.157511 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e099a01c-83eb-421e-a2f6-79e3fe63f042 admin admin] PATH_INFO: `/v3/projects/0912369993c943529c87b2f8d6e02d1e/users/1f61a795f30a420c83aa65cebab57c60/roles/e0ab41d8ae22474688b5361bcc75e0f0` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:51.166122 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e099a01c-83eb-421e-a2f6-79e3fe63f042 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0912369993c943529c87b2f8d6e02d1e, user_id=1f61a795f30a420c83aa65cebab57c60, role_id=e0ab41d8ae22474688b5361bcc75e0f0)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:51.179360 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e099a01c-83eb-421e-a2f6-79e3fe63f042 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:51.217111 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb5b667-c2d5-4072-9b8e-011ac7a18001 admin admin] RBAC: Authorizing `identity:create_grant(project_id=982ddc3647c446de8a81f3dbce35e35f, user_id=13c49de541c749779489b9e8314736cf, role_id=4eea39e24b06431aa357053fe7907a3d)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:51.219989 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb5b667-c2d5-4072-9b8e-011ac7a18001 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:51.231084 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 920/1822] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:15:51 2022] PUT /identity/v3/projects/0912369993c943529c87b2f8d6e02d1e/users/1f61a795f30a420c83aa65cebab57c60/roles/e0ab41d8ae22474688b5361bcc75e0f0 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:15:51.251543 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7467ba50-edef-44a5-bcaa-aa268012aad8 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:51.252151 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7467ba50-edef-44a5-bcaa-aa268012aad8 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:51.252608 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7467ba50-edef-44a5-bcaa-aa268012aad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:51.257934 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 903/1823] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:15:50 2022] PUT /identity/v3/projects/982ddc3647c446de8a81f3dbce35e35f/users/13c49de541c749779489b9e8314736cf/roles/4eea39e24b06431aa357053fe7907a3d => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:15:51.279245 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1784ef0-c150-492b-8e05-83a44e05c3d8 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:51.280060 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1784ef0-c150-492b-8e05-83a44e05c3d8 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:51.280451 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1784ef0-c150-492b-8e05-83a44e05c3d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:51.324162 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-c1784ef0-c150-492b-8e05-83a44e05c3d8 None None] MFA Rules not processed for user `13c49de541c749779489b9e8314736cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:15:51.326029 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-c1784ef0-c150-492b-8e05-83a44e05c3d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:51.343473 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:15:51.378184 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-7467ba50-edef-44a5-bcaa-aa268012aad8 None None] MFA Rules not processed for user `1f61a795f30a420c83aa65cebab57c60`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:15:51.382250 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-7467ba50-edef-44a5-bcaa-aa268012aad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:51.439201 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:15:51.452405 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 904/1824] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:15:51 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:15:51.453795 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1784ef0-c150-492b-8e05-83a44e05c3d8 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:51.461938 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-c1784ef0-c150-492b-8e05-83a44e05c3d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:51.593755 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1784ef0-c150-492b-8e05-83a44e05c3d8 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:51.600676 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b0fc5f-110d-4d81-bb31-2063224f6cf2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:51.602345 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b0fc5f-110d-4d81-bb31-2063224f6cf2 service neutron] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:51.602700 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b0fc5f-110d-4d81-bb31-2063224f6cf2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:51.603059 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b0fc5f-110d-4d81-bb31-2063224f6cf2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:51.715318 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 921/1825] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:15:51 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 466 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:15:51.716556 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7467ba50-edef-44a5-bcaa-aa268012aad8 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:51.724660 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-7467ba50-edef-44a5-bcaa-aa268012aad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:51.884088 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b0fc5f-110d-4d81-bb31-2063224f6cf2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:51.887239 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b0fc5f-110d-4d81-bb31-2063224f6cf2 service neutron] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:51.923548 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 905/1826] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:15:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:15:51.928944 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b0fc5f-110d-4d81-bb31-2063224f6cf2 service neutron] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:51.929179 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7467ba50-edef-44a5-bcaa-aa268012aad8 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:51.933315 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-16b0fc5f-110d-4d81-bb31-2063224f6cf2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:51.940983 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1370f8fa-bbf7-4e5e-bbb0-460174d821bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df1bca993bc142908cc7cf85eabd2b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:51.945452 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1370f8fa-bbf7-4e5e-bbb0-460174d821bc service cinder] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:51.946586 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1370f8fa-bbf7-4e5e-bbb0-460174d821bc service cinder] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:51.947755 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1370f8fa-bbf7-4e5e-bbb0-460174d821bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:51.973824 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1370f8fa-bbf7-4e5e-bbb0-460174d821bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:51.980040 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1370f8fa-bbf7-4e5e-bbb0-460174d821bc service cinder] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:51.983522 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b0fc5f-110d-4d81-bb31-2063224f6cf2 service neutron] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:52.002910 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e92fad-20c8-45fb-a169-69f4dfad75c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df1bca993bc142908cc7cf85eabd2b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:52.003784 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e92fad-20c8-45fb-a169-69f4dfad75c5 service cinder] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:52.004308 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e92fad-20c8-45fb-a169-69f4dfad75c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:52.007746 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e92fad-20c8-45fb-a169-69f4dfad75c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:52.008257 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 922/1827] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:15:51 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:15:52.018567 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-12e92fad-20c8-45fb-a169-69f4dfad75c5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:52.126041 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1370f8fa-bbf7-4e5e-bbb0-460174d821bc service cinder] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:52.150320 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1370f8fa-bbf7-4e5e-bbb0-460174d821bc service cinder] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:52.184415 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a7e34d-21e7-4c4a-861b-147a305f0e4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:52.187426 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a7e34d-21e7-4c4a-861b-147a305f0e4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:52.187857 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a7e34d-21e7-4c4a-861b-147a305f0e4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:52.194953 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a7e34d-21e7-4c4a-861b-147a305f0e4e admin admin] PATH_INFO: `/v3/users/13c49de541c749779489b9e8314736cf` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:52.198674 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a7e34d-21e7-4c4a-861b-147a305f0e4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=13c49de541c749779489b9e8314736cf)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:52.204687 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a7e34d-21e7-4c4a-861b-147a305f0e4e admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:52.286887 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e92fad-20c8-45fb-a169-69f4dfad75c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:52.292179 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e92fad-20c8-45fb-a169-69f4dfad75c5 service cinder] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:52.310647 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:15:52.329163 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 906/1828] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:15:51 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:15:52.343425 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-34a7e34d-21e7-4c4a-861b-147a305f0e4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13c49de541c749779489b9e8314736cf'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:52.345078 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-34a7e34d-21e7-4c4a-861b-147a305f0e4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13c49de541c749779489b9e8314736cf'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:52.369173 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-34a7e34d-21e7-4c4a-861b-147a305f0e4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13c49de541c749779489b9e8314736cf'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:52.370278 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-34a7e34d-21e7-4c4a-861b-147a305f0e4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13c49de541c749779489b9e8314736cf'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:52.382154 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-34a7e34d-21e7-4c4a-861b-147a305f0e4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13c49de541c749779489b9e8314736cf'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:52.411178 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 923/1829] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:15:52 2022] DELETE /identity/v3/users/13c49de541c749779489b9e8314736cf => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 20:15:52.427319 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b21fd7-5054-4f6a-8ac3-085ec5e698da None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:52.427628 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b21fd7-5054-4f6a-8ac3-085ec5e698da None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:52.427950 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b21fd7-5054-4f6a-8ac3-085ec5e698da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:52.477981 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-59b21fd7-5054-4f6a-8ac3-085ec5e698da None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:15:52.479321 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-59b21fd7-5054-4f6a-8ac3-085ec5e698da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:52.698445 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 907/1830] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:15:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:15:52.748078 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a7e34d-21e7-4c4a-861b-147a305f0e4e admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:52.749892 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-34a7e34d-21e7-4c4a-861b-147a305f0e4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:52.924874 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:15:53.112193 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a7e34d-21e7-4c4a-861b-147a305f0e4e admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:53.141654 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f222af09-9adf-497e-a440-0ddb742cd621 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:53.142225 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f222af09-9adf-497e-a440-0ddb742cd621 service neutron] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:53.142512 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f222af09-9adf-497e-a440-0ddb742cd621 service neutron] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:53.142796 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f222af09-9adf-497e-a440-0ddb742cd621 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:53.159170 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f222af09-9adf-497e-a440-0ddb742cd621 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:53.163473 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f222af09-9adf-497e-a440-0ddb742cd621 service neutron] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:53.188994 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 924/1831] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:15:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:15:54.699684 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b21fd7-5054-4f6a-8ac3-085ec5e698da None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:54.702170 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-59b21fd7-5054-4f6a-8ac3-085ec5e698da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:54.776587 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b21fd7-5054-4f6a-8ac3-085ec5e698da None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:54.791955 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fb3628-3a97-4b01-9476-ae10391291fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:54.792542 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fb3628-3a97-4b01-9476-ae10391291fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:54.792893 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fb3628-3a97-4b01-9476-ae10391291fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:54.793350 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fb3628-3a97-4b01-9476-ae10391291fc admin admin] PATH_INFO: `/v3/projects/957fd609924b4f05acfe702c31757437` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:54.802357 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fb3628-3a97-4b01-9476-ae10391291fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=957fd609924b4f05acfe702c31757437)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:54.807576 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fb3628-3a97-4b01-9476-ae10391291fc admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:55.096360 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-19fb3628-3a97-4b01-9476-ae10391291fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '957fd609924b4f05acfe702c31757437', 'request_id': 'req-19fb3628-3a97-4b01-9476-ae10391291fc'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:55.111790 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-19fb3628-3a97-4b01-9476-ae10391291fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '957fd609924b4f05acfe702c31757437', 'request_id': 'req-19fb3628-3a97-4b01-9476-ae10391291fc'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:55.131292 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-19fb3628-3a97-4b01-9476-ae10391291fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '957fd609924b4f05acfe702c31757437', 'request_id': 'req-19fb3628-3a97-4b01-9476-ae10391291fc'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:55.171482 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-19fb3628-3a97-4b01-9476-ae10391291fc admin admin] The token cache is being invalidate because project 957fd609924b4f05acfe702c31757437 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81771) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:15:55.172071 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-19fb3628-3a97-4b01-9476-ae10391291fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:55.173829 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-19fb3628-3a97-4b01-9476-ae10391291fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:55.183500 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f222af09-9adf-497e-a440-0ddb742cd621 service neutron] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:55.184105 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 908/1832] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:15:54 2022] DELETE /identity/v3/projects/957fd609924b4f05acfe702c31757437 => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:15:55.223636 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f222af09-9adf-497e-a440-0ddb742cd621 service neutron] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:55.325464 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fb3628-3a97-4b01-9476-ae10391291fc admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:55.332846 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-19fb3628-3a97-4b01-9476-ae10391291fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:55.443775 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3e9b5a-565e-4ca1-9397-2c9dbaa9b688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:55.444912 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3e9b5a-565e-4ca1-9397-2c9dbaa9b688 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:55.445414 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3e9b5a-565e-4ca1-9397-2c9dbaa9b688 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:55.445937 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3e9b5a-565e-4ca1-9397-2c9dbaa9b688 admin admin] PATH_INFO: `/v3/users/66f82d33ee55445fa36af756a2c9f499` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:55.449083 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3e9b5a-565e-4ca1-9397-2c9dbaa9b688 admin admin] RBAC: Authorizing `identity:delete_user(user_id=66f82d33ee55445fa36af756a2c9f499)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:55.452972 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3e9b5a-565e-4ca1-9397-2c9dbaa9b688 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:55.587026 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fb3628-3a97-4b01-9476-ae10391291fc admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:55.595117 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae32436-116c-4691-b6e6-090592b7249b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:55.596092 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae32436-116c-4691-b6e6-090592b7249b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:55.596250 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae32436-116c-4691-b6e6-090592b7249b admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:55.596410 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae32436-116c-4691-b6e6-090592b7249b admin admin] PATH_INFO: `/v3/projects/982ddc3647c446de8a81f3dbce35e35f` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:55.598382 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae32436-116c-4691-b6e6-090592b7249b admin admin] RBAC: Authorizing `identity:delete_project(project_id=982ddc3647c446de8a81f3dbce35e35f)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:55.602497 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae32436-116c-4691-b6e6-090592b7249b admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:56.223598 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:15:56.262085 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-ed3e9b5a-565e-4ca1-9397-2c9dbaa9b688 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66f82d33ee55445fa36af756a2c9f499'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:56.263081 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-ed3e9b5a-565e-4ca1-9397-2c9dbaa9b688 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66f82d33ee55445fa36af756a2c9f499'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:56.273159 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-5ae32436-116c-4691-b6e6-090592b7249b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '982ddc3647c446de8a81f3dbce35e35f', 'request_id': 'req-5ae32436-116c-4691-b6e6-090592b7249b'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:56.281703 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-5ae32436-116c-4691-b6e6-090592b7249b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '982ddc3647c446de8a81f3dbce35e35f', 'request_id': 'req-5ae32436-116c-4691-b6e6-090592b7249b'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:56.282186 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-ed3e9b5a-565e-4ca1-9397-2c9dbaa9b688 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66f82d33ee55445fa36af756a2c9f499'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:56.282948 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-ed3e9b5a-565e-4ca1-9397-2c9dbaa9b688 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66f82d33ee55445fa36af756a2c9f499'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:56.294163 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-5ae32436-116c-4691-b6e6-090592b7249b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '982ddc3647c446de8a81f3dbce35e35f', 'request_id': 'req-5ae32436-116c-4691-b6e6-090592b7249b'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:56.299629 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-ed3e9b5a-565e-4ca1-9397-2c9dbaa9b688 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66f82d33ee55445fa36af756a2c9f499'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:56.308391 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-5ae32436-116c-4691-b6e6-090592b7249b admin admin] The token cache is being invalidate because project 982ddc3647c446de8a81f3dbce35e35f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81771) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:15:56.308790 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-5ae32436-116c-4691-b6e6-090592b7249b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:56.310078 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-5ae32436-116c-4691-b6e6-090592b7249b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:56.315967 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 909/1833] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:15:55 2022] DELETE /identity/v3/projects/982ddc3647c446de8a81f3dbce35e35f => generated 0 bytes in 989 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:15:56.315967 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db602f41-e4fe-4e6e-834d-bda7c8e4534d None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:56.315967 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db602f41-e4fe-4e6e-834d-bda7c8e4534d None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:56.315967 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db602f41-e4fe-4e6e-834d-bda7c8e4534d None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:56.319184 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 910/1834] 10.222.0.20 () {54 vars in 921 bytes} [Tue Sep 6 20:15:56 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:15:56.321451 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96114492-be9c-47e1-bbd9-8a3135d6d4b2 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:56.321956 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96114492-be9c-47e1-bbd9-8a3135d6d4b2 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:56.322446 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96114492-be9c-47e1-bbd9-8a3135d6d4b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:56.329058 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 925/1835] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:15:55 2022] DELETE /identity/v3/users/66f82d33ee55445fa36af756a2c9f499 => generated 0 bytes in 1146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 20:15:56.349780 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fbe89b-cba9-430e-87c3-d536e2a5c420 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:56.350942 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fbe89b-cba9-430e-87c3-d536e2a5c420 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:56.352574 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fbe89b-cba9-430e-87c3-d536e2a5c420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:56.364576 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-96114492-be9c-47e1-bbd9-8a3135d6d4b2 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:15:56.367077 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-96114492-be9c-47e1-bbd9-8a3135d6d4b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:56.399955 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-79fbe89b-cba9-430e-87c3-d536e2a5c420 None None] MFA Rules not processed for user `5776cbf4f6714b2083c3ebdd6d98cc87`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:15:56.402508 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-79fbe89b-cba9-430e-87c3-d536e2a5c420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:56.511572 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 911/1836] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:15:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:15:56.527204 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42dcb68-4ea5-4f42-842e-51c06450f057 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:56.527610 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42dcb68-4ea5-4f42-842e-51c06450f057 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:56.528623 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42dcb68-4ea5-4f42-842e-51c06450f057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:56.537705 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:15:56.588678 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-f42dcb68-4ea5-4f42-842e-51c06450f057 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:15:56.591659 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-f42dcb68-4ea5-4f42-842e-51c06450f057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:56.604371 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 926/1837] 10.222.0.20 () {58 vars in 1018 bytes} [Tue Sep 6 20:15:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:15:56.611204 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 912/1838] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:15:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:15:56.617686 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fbe89b-cba9-430e-87c3-d536e2a5c420 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:56.635863 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42dcb68-4ea5-4f42-842e-51c06450f057 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:56.637303 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-f42dcb68-4ea5-4f42-842e-51c06450f057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:56.642570 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fbe89b-cba9-430e-87c3-d536e2a5c420 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:56.661363 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1c0fb0-2fe3-48c2-8cad-5e751b540d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:56.662090 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1c0fb0-2fe3-48c2-8cad-5e751b540d9d admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:56.662379 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1c0fb0-2fe3-48c2-8cad-5e751b540d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:56.662654 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1c0fb0-2fe3-48c2-8cad-5e751b540d9d admin admin] PATH_INFO: `/v3/domains` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:56.663393 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1c0fb0-2fe3-48c2-8cad-5e751b540d9d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:15:56.663702 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1c0fb0-2fe3-48c2-8cad-5e751b540d9d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:56.665540 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1c0fb0-2fe3-48c2-8cad-5e751b540d9d admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:56.685657 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 927/1839] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 6 20:15:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 20:15:56.691453 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1c0fb0-2fe3-48c2-8cad-5e751b540d9d admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:56.692506 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42dcb68-4ea5-4f42-842e-51c06450f057 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:56.700563 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4d475e-fc45-4b42-a02d-680bc65664db service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5776cbf4f6714b2083c3ebdd6d98cc87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:56.702715 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4d475e-fc45-4b42-a02d-680bc65664db service nova] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:56.703104 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4d475e-fc45-4b42-a02d-680bc65664db service nova] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:56.703483 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4d475e-fc45-4b42-a02d-680bc65664db service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:56.714634 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1c0fb0-2fe3-48c2-8cad-5e751b540d9d admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:56.718639 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4d475e-fc45-4b42-a02d-680bc65664db service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:56.720395 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4d475e-fc45-4b42-a02d-680bc65664db service nova] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:56.721676 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f2a452-d0ff-44c5-88b2-15695dcc7c11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:56.722521 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f2a452-d0ff-44c5-88b2-15695dcc7c11 admin admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:56.722863 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f2a452-d0ff-44c5-88b2-15695dcc7c11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:56.723153 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f2a452-d0ff-44c5-88b2-15695dcc7c11 admin admin] PATH_INFO: `/v3/projects` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:56.723824 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f2a452-d0ff-44c5-88b2-15695dcc7c11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:56.725960 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f2a452-d0ff-44c5-88b2-15695dcc7c11 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:56.767000 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 928/1840] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:15:56 2022] POST /identity/v3/projects => generated 371 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:15:56.776228 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f2a452-d0ff-44c5-88b2-15695dcc7c11 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:56.789734 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 913/1841] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:15:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 20:15:56.877970 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f2a452-d0ff-44c5-88b2-15695dcc7c11 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:56.887683 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00d9a58-db9a-499e-ba4f-fe8afdb3267d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:56.888755 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00d9a58-db9a-499e-ba4f-fe8afdb3267d admin admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:56.889048 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00d9a58-db9a-499e-ba4f-fe8afdb3267d admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:56.889421 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00d9a58-db9a-499e-ba4f-fe8afdb3267d admin admin] PATH_INFO: `/v3/users` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:56.890394 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00d9a58-db9a-499e-ba4f-fe8afdb3267d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:56.893397 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00d9a58-db9a-499e-ba4f-fe8afdb3267d admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:56.939917 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 929/1842] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:15:56 2022] POST /identity/v3/users => generated 286 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:15:56.942683 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4d475e-fc45-4b42-a02d-680bc65664db service nova] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:56.990922 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4d475e-fc45-4b42-a02d-680bc65664db service nova] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:56.999696 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fff73b2-ad90-41b3-bf3f-c34eeb896094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:57.002235 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fff73b2-ad90-41b3-bf3f-c34eeb896094 admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:57.003023 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fff73b2-ad90-41b3-bf3f-c34eeb896094 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:57.004032 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fff73b2-ad90-41b3-bf3f-c34eeb896094 admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:57.005214 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fff73b2-ad90-41b3-bf3f-c34eeb896094 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:57.007036 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fff73b2-ad90-41b3-bf3f-c34eeb896094 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:57.025891 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 914/1843] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:15:56 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:15:57.028129 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00d9a58-db9a-499e-ba4f-fe8afdb3267d admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:57.041798 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00d9a58-db9a-499e-ba4f-fe8afdb3267d admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:57.050689 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02dbc1e-c1eb-4e9e-bc10-3e6aa28a06ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:57.051503 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02dbc1e-c1eb-4e9e-bc10-3e6aa28a06ac admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:57.051793 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02dbc1e-c1eb-4e9e-bc10-3e6aa28a06ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:57.052138 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02dbc1e-c1eb-4e9e-bc10-3e6aa28a06ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:57.052743 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02dbc1e-c1eb-4e9e-bc10-3e6aa28a06ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:57.054723 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02dbc1e-c1eb-4e9e-bc10-3e6aa28a06ac admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:57.072905 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 930/1844] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:15:57 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:15:57.074945 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fff73b2-ad90-41b3-bf3f-c34eeb896094 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:57.090445 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fff73b2-ad90-41b3-bf3f-c34eeb896094 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:57.100717 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff2107f-1ed1-4dc1-9f10-da2c5435df09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:57.103073 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff2107f-1ed1-4dc1-9f10-da2c5435df09 admin admin] REQUEST_METHOD: `PUT` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:57.103424 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff2107f-1ed1-4dc1-9f10-da2c5435df09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:57.103827 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff2107f-1ed1-4dc1-9f10-da2c5435df09 admin admin] PATH_INFO: `/v3/projects/72ea080ba949431b9988b4caae9dc61a/users/3664ee4dc08844a8a7a31005894fc341/roles/4eea39e24b06431aa357053fe7907a3d` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:57.143688 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff2107f-1ed1-4dc1-9f10-da2c5435df09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72ea080ba949431b9988b4caae9dc61a, user_id=3664ee4dc08844a8a7a31005894fc341, role_id=4eea39e24b06431aa357053fe7907a3d)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:57.149899 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff2107f-1ed1-4dc1-9f10-da2c5435df09 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:57.174506 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 915/1845] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:15:57 2022] PUT /identity/v3/projects/72ea080ba949431b9988b4caae9dc61a/users/3664ee4dc08844a8a7a31005894fc341/roles/4eea39e24b06431aa357053fe7907a3d => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:15:57.189156 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02dbc1e-c1eb-4e9e-bc10-3e6aa28a06ac admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:57.197775 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5252eb-fbe3-4060-b4d8-c53a30dc5c23 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:57.198360 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5252eb-fbe3-4060-b4d8-c53a30dc5c23 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:57.198897 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5252eb-fbe3-4060-b4d8-c53a30dc5c23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:57.200420 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-e02dbc1e-c1eb-4e9e-bc10-3e6aa28a06ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:57.228613 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02dbc1e-c1eb-4e9e-bc10-3e6aa28a06ac admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:57.281688 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-cf5252eb-fbe3-4060-b4d8-c53a30dc5c23 None None] MFA Rules not processed for user `3664ee4dc08844a8a7a31005894fc341`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:15:57.283930 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-cf5252eb-fbe3-4060-b4d8-c53a30dc5c23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:57.374124 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9947bd71-1240-4719-8217-4c6f01f09eb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:57.387320 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9947bd71-1240-4719-8217-4c6f01f09eb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:57.389444 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9947bd71-1240-4719-8217-4c6f01f09eb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:57.390366 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9947bd71-1240-4719-8217-4c6f01f09eb3 admin admin] PATH_INFO: `/v3/projects/919bf1f2d52e4526b0a1edcda5a2dd3d` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:57.394112 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9947bd71-1240-4719-8217-4c6f01f09eb3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=919bf1f2d52e4526b0a1edcda5a2dd3d)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:57.399936 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9947bd71-1240-4719-8217-4c6f01f09eb3 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:57.423326 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:15:57.461745 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 916/1846] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:15:57 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:15:57.478436 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:15:57.491861 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5252eb-fbe3-4060-b4d8-c53a30dc5c23 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:57.494462 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-cf5252eb-fbe3-4060-b4d8-c53a30dc5c23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:57.514080 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-9947bd71-1240-4719-8217-4c6f01f09eb3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '919bf1f2d52e4526b0a1edcda5a2dd3d', 'request_id': 'req-9947bd71-1240-4719-8217-4c6f01f09eb3'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:57.527617 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-9947bd71-1240-4719-8217-4c6f01f09eb3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '919bf1f2d52e4526b0a1edcda5a2dd3d', 'request_id': 'req-9947bd71-1240-4719-8217-4c6f01f09eb3'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:57.536151 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-9947bd71-1240-4719-8217-4c6f01f09eb3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '919bf1f2d52e4526b0a1edcda5a2dd3d', 'request_id': 'req-9947bd71-1240-4719-8217-4c6f01f09eb3'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:57.552470 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-9947bd71-1240-4719-8217-4c6f01f09eb3 admin admin] The token cache is being invalidate because project 919bf1f2d52e4526b0a1edcda5a2dd3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81772) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:15:57.552821 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-9947bd71-1240-4719-8217-4c6f01f09eb3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:57.554024 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-9947bd71-1240-4719-8217-4c6f01f09eb3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:57.562205 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 931/1847] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:15:57 2022] DELETE /identity/v3/projects/919bf1f2d52e4526b0a1edcda5a2dd3d => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:15:57.562205 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9947bd71-1240-4719-8217-4c6f01f09eb3 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:57.562205 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-9947bd71-1240-4719-8217-4c6f01f09eb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:57.629775 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5252eb-fbe3-4060-b4d8-c53a30dc5c23 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:57.670315 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9947bd71-1240-4719-8217-4c6f01f09eb3 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:57.678073 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc68c8e-5137-4e25-8382-c0be40af5dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:57.679190 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc68c8e-5137-4e25-8382-c0be40af5dce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:57.679643 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc68c8e-5137-4e25-8382-c0be40af5dce admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:57.680151 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc68c8e-5137-4e25-8382-c0be40af5dce admin admin] PATH_INFO: `/v3/users/162a61788d3d426aa9d5e39e17dd2e90` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:57.682290 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc68c8e-5137-4e25-8382-c0be40af5dce admin admin] RBAC: Authorizing `identity:delete_user(user_id=162a61788d3d426aa9d5e39e17dd2e90)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:57.686570 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc68c8e-5137-4e25-8382-c0be40af5dce admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:57.700007 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b409fe-b287-42e7-8662-7bb0dc5e4321 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df1bca993bc142908cc7cf85eabd2b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:57.700641 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b409fe-b287-42e7-8662-7bb0dc5e4321 service cinder] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:57.700937 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b409fe-b287-42e7-8662-7bb0dc5e4321 service cinder] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:57.701244 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b409fe-b287-42e7-8662-7bb0dc5e4321 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:57.703227 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-41b409fe-b287-42e7-8662-7bb0dc5e4321 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:57.776117 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-efc68c8e-5137-4e25-8382-c0be40af5dce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '162a61788d3d426aa9d5e39e17dd2e90'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:57.778259 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-efc68c8e-5137-4e25-8382-c0be40af5dce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '162a61788d3d426aa9d5e39e17dd2e90'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:57.801785 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b409fe-b287-42e7-8662-7bb0dc5e4321 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:57.803747 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b409fe-b287-42e7-8662-7bb0dc5e4321 service cinder] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:57.808083 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-efc68c8e-5137-4e25-8382-c0be40af5dce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '162a61788d3d426aa9d5e39e17dd2e90'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:57.813540 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-efc68c8e-5137-4e25-8382-c0be40af5dce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '162a61788d3d426aa9d5e39e17dd2e90'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:57.817107 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 917/1848] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:15:57 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 20:15:57.830139 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-efc68c8e-5137-4e25-8382-c0be40af5dce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '162a61788d3d426aa9d5e39e17dd2e90'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:57.863179 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b409fe-b287-42e7-8662-7bb0dc5e4321 service cinder] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:57.864768 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-41b409fe-b287-42e7-8662-7bb0dc5e4321 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:57.870342 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 932/1849] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:15:57 2022] DELETE /identity/v3/users/162a61788d3d426aa9d5e39e17dd2e90 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:15:57.919099 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b409fe-b287-42e7-8662-7bb0dc5e4321 service cinder] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:58.195479 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e462105-6093-4b2e-939d-7330553768d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:58.200987 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e462105-6093-4b2e-939d-7330553768d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:58.203390 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e462105-6093-4b2e-939d-7330553768d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:58.209951 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e462105-6093-4b2e-939d-7330553768d8 admin admin] PATH_INFO: `/v3/users/3664ee4dc08844a8a7a31005894fc341` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:58.220115 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e462105-6093-4b2e-939d-7330553768d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3664ee4dc08844a8a7a31005894fc341)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:58.231666 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e462105-6093-4b2e-939d-7330553768d8 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:58.247686 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:15:58.264207 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc68c8e-5137-4e25-8382-c0be40af5dce admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:58.268767 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-efc68c8e-5137-4e25-8382-c0be40af5dce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:58.427832 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:15:58.462895 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-6e462105-6093-4b2e-939d-7330553768d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3664ee4dc08844a8a7a31005894fc341'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:58.479569 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-6e462105-6093-4b2e-939d-7330553768d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3664ee4dc08844a8a7a31005894fc341'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:58.482778 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-6e462105-6093-4b2e-939d-7330553768d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3664ee4dc08844a8a7a31005894fc341'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:58.483811 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-6e462105-6093-4b2e-939d-7330553768d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3664ee4dc08844a8a7a31005894fc341'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:58.491353 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc68c8e-5137-4e25-8382-c0be40af5dce admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:58.503328 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029cad56-088a-4816-a999-b2f770c0f7d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df1bca993bc142908cc7cf85eabd2b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:58.507979 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029cad56-088a-4816-a999-b2f770c0f7d4 service cinder] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:58.508195 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029cad56-088a-4816-a999-b2f770c0f7d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:58.509329 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029cad56-088a-4816-a999-b2f770c0f7d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:58.512655 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-029cad56-088a-4816-a999-b2f770c0f7d4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:58.530994 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-6e462105-6093-4b2e-939d-7330553768d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3664ee4dc08844a8a7a31005894fc341'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:15:58.586803 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 918/1850] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:15:57 2022] DELETE /identity/v3/users/3664ee4dc08844a8a7a31005894fc341 => generated 0 bytes in 724 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:15:58.597130 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0769f920-b3bf-4705-b509-af507459f40f None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:58.598698 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0769f920-b3bf-4705-b509-af507459f40f None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:58.599558 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0769f920-b3bf-4705-b509-af507459f40f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:58.654729 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-0769f920-b3bf-4705-b509-af507459f40f None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:15:58.656839 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-0769f920-b3bf-4705-b509-af507459f40f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:58.689509 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029cad56-088a-4816-a999-b2f770c0f7d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:58.697253 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029cad56-088a-4816-a999-b2f770c0f7d4 service cinder] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:58.734048 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 933/1851] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:15:58 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:15:58.882931 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 919/1852] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:15:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:15:59.003751 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029cad56-088a-4816-a999-b2f770c0f7d4 service cinder] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:15:59.006970 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-029cad56-088a-4816-a999-b2f770c0f7d4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:15:59.174622 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029cad56-088a-4816-a999-b2f770c0f7d4 service cinder] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:15:59.303483 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a444f1-48e7-4224-9bba-b75d4bde02c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:15:59.304943 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a444f1-48e7-4224-9bba-b75d4bde02c6 service neutron] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:15:59.305551 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a444f1-48e7-4224-9bba-b75d4bde02c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:15:59.306362 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a444f1-48e7-4224-9bba-b75d4bde02c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:15:59.328287 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a444f1-48e7-4224-9bba-b75d4bde02c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:15:59.332048 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a444f1-48e7-4224-9bba-b75d4bde02c6 service neutron] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:15:59.355032 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 934/1853] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:15:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:16:00.180602 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0769f920-b3bf-4705-b509-af507459f40f None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:16:00.183352 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-0769f920-b3bf-4705-b509-af507459f40f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:16:00.222106 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0769f920-b3bf-4705-b509-af507459f40f None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:16:00.252264 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6bc7a8-4840-40ae-9765-8185412e0135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:16:00.263278 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6bc7a8-4840-40ae-9765-8185412e0135 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:00.263908 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6bc7a8-4840-40ae-9765-8185412e0135 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:00.264892 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6bc7a8-4840-40ae-9765-8185412e0135 admin admin] PATH_INFO: `/v3/projects/11622d6408c2496e835711e4e1175e3f` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:00.279333 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6bc7a8-4840-40ae-9765-8185412e0135 admin admin] RBAC: Authorizing `identity:delete_project(project_id=11622d6408c2496e835711e4e1175e3f)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:16:00.294709 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6bc7a8-4840-40ae-9765-8185412e0135 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:16:00.470362 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-2e6bc7a8-4840-40ae-9765-8185412e0135 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11622d6408c2496e835711e4e1175e3f', 'request_id': 'req-2e6bc7a8-4840-40ae-9765-8185412e0135'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:16:00.491689 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-2e6bc7a8-4840-40ae-9765-8185412e0135 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11622d6408c2496e835711e4e1175e3f', 'request_id': 'req-2e6bc7a8-4840-40ae-9765-8185412e0135'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:16:00.514605 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-2e6bc7a8-4840-40ae-9765-8185412e0135 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11622d6408c2496e835711e4e1175e3f', 'request_id': 'req-2e6bc7a8-4840-40ae-9765-8185412e0135'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:16:00.552695 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-2e6bc7a8-4840-40ae-9765-8185412e0135 admin admin] The token cache is being invalidate because project 11622d6408c2496e835711e4e1175e3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81771) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:16:00.553269 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-2e6bc7a8-4840-40ae-9765-8185412e0135 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:16:00.554607 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-2e6bc7a8-4840-40ae-9765-8185412e0135 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:16:00.558676 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 920/1854] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:16:00 2022] DELETE /identity/v3/projects/11622d6408c2496e835711e4e1175e3f => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:16:00.564428 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6abfee-1695-4d6e-a02b-28a2340af674 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:00.565520 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6abfee-1695-4d6e-a02b-28a2340af674 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:00.566164 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6abfee-1695-4d6e-a02b-28a2340af674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:00.603321 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-7c6abfee-1695-4d6e-a02b-28a2340af674 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:16:00.606128 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-7c6abfee-1695-4d6e-a02b-28a2340af674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:16:00.690055 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:16:00.732497 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 935/1855] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:16:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:16:00.742488 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98978267-de10-4960-bd40-f69a53e29e0d None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:00.742866 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98978267-de10-4960-bd40-f69a53e29e0d None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:00.743431 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98978267-de10-4960-bd40-f69a53e29e0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:00.771635 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:16:00.810391 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-98978267-de10-4960-bd40-f69a53e29e0d None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:16:00.814636 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-98978267-de10-4960-bd40-f69a53e29e0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:16:00.837752 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 921/1856] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:16:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:16:00.843344 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6abfee-1695-4d6e-a02b-28a2340af674 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:16:00.858965 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6abfee-1695-4d6e-a02b-28a2340af674 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:16:00.865058 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c145222-6e80-4653-b898-e68fcb328d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:16:00.865355 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c145222-6e80-4653-b898-e68fcb328d62 admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:00.865793 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c145222-6e80-4653-b898-e68fcb328d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:00.866036 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c145222-6e80-4653-b898-e68fcb328d62 admin admin] PATH_INFO: `/v3/domains` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:00.866753 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c145222-6e80-4653-b898-e68fcb328d62 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:16:00.867041 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c145222-6e80-4653-b898-e68fcb328d62 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:16:00.869304 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c145222-6e80-4653-b898-e68fcb328d62 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:16:00.892753 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 936/1857] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 6 20:16:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:16:00.895141 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98978267-de10-4960-bd40-f69a53e29e0d None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:16:00.922995 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98978267-de10-4960-bd40-f69a53e29e0d None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:16:00.936958 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867da156-f1cf-4667-9c06-7a734cbef0d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:16:00.938288 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867da156-f1cf-4667-9c06-7a734cbef0d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:00.938569 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867da156-f1cf-4667-9c06-7a734cbef0d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:00.939039 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867da156-f1cf-4667-9c06-7a734cbef0d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:00.940314 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867da156-f1cf-4667-9c06-7a734cbef0d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:16:00.948525 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867da156-f1cf-4667-9c06-7a734cbef0d8 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:16:01.013540 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 922/1858] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:16:00 2022] POST /identity/v3/projects => generated 371 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:16:01.019399 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c145222-6e80-4653-b898-e68fcb328d62 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:16:01.034277 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c145222-6e80-4653-b898-e68fcb328d62 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:16:01.153845 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a5ea2a-3728-4d79-8d98-9a58a5fb762a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:16:01.155044 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a5ea2a-3728-4d79-8d98-9a58a5fb762a admin admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:01.155670 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a5ea2a-3728-4d79-8d98-9a58a5fb762a admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:01.156085 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a5ea2a-3728-4d79-8d98-9a58a5fb762a admin admin] PATH_INFO: `/v3/users` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:01.157426 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a5ea2a-3728-4d79-8d98-9a58a5fb762a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:16:01.161725 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a5ea2a-3728-4d79-8d98-9a58a5fb762a admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:16:01.198944 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 937/1859] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:16:01 2022] POST /identity/v3/users => generated 286 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:16:01.204880 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867da156-f1cf-4667-9c06-7a734cbef0d8 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:16:01.207326 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a5ea2a-3728-4d79-8d98-9a58a5fb762a admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:16:01.210746 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-36a5ea2a-3728-4d79-8d98-9a58a5fb762a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:16:01.223687 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867da156-f1cf-4667-9c06-7a734cbef0d8 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:16:01.230088 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a5ea2a-3728-4d79-8d98-9a58a5fb762a admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:16:01.240178 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2760501f-ca12-446c-a259-5a7ffe802673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:16:01.242189 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2760501f-ca12-446c-a259-5a7ffe802673 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:01.242578 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2760501f-ca12-446c-a259-5a7ffe802673 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:01.243483 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2760501f-ca12-446c-a259-5a7ffe802673 admin admin] PATH_INFO: `/v3/projects/72ea080ba949431b9988b4caae9dc61a` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:01.245480 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2760501f-ca12-446c-a259-5a7ffe802673 admin admin] RBAC: Authorizing `identity:delete_project(project_id=72ea080ba949431b9988b4caae9dc61a)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:16:01.255152 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2760501f-ca12-446c-a259-5a7ffe802673 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:16:01.264001 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d274998e-2e3a-46a6-9fba-92e7d38879eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:16:01.265168 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d274998e-2e3a-46a6-9fba-92e7d38879eb admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:01.265707 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d274998e-2e3a-46a6-9fba-92e7d38879eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:01.266220 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d274998e-2e3a-46a6-9fba-92e7d38879eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:01.267210 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d274998e-2e3a-46a6-9fba-92e7d38879eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:16:01.270475 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d274998e-2e3a-46a6-9fba-92e7d38879eb admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:16:01.324611 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 923/1860] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:16:01 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:16:01.329146 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d274998e-2e3a-46a6-9fba-92e7d38879eb admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:16:01.379836 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d274998e-2e3a-46a6-9fba-92e7d38879eb admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:16:01.422620 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-2760501f-ca12-446c-a259-5a7ffe802673 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72ea080ba949431b9988b4caae9dc61a', 'request_id': 'req-2760501f-ca12-446c-a259-5a7ffe802673'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:16:01.439089 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-2760501f-ca12-446c-a259-5a7ffe802673 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72ea080ba949431b9988b4caae9dc61a', 'request_id': 'req-2760501f-ca12-446c-a259-5a7ffe802673'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:16:01.451874 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-2760501f-ca12-446c-a259-5a7ffe802673 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72ea080ba949431b9988b4caae9dc61a', 'request_id': 'req-2760501f-ca12-446c-a259-5a7ffe802673'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:16:01.468326 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-2760501f-ca12-446c-a259-5a7ffe802673 admin admin] The token cache is being invalidate because project 72ea080ba949431b9988b4caae9dc61a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81772) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:16:01.468755 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-2760501f-ca12-446c-a259-5a7ffe802673 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:16:01.469491 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-2760501f-ca12-446c-a259-5a7ffe802673 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:16:01.476559 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 938/1861] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:16:01 2022] DELETE /identity/v3/projects/72ea080ba949431b9988b4caae9dc61a => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:16:01.478441 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35efd0c2-e1b2-4b71-992e-81fc45525153 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:01.478900 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35efd0c2-e1b2-4b71-992e-81fc45525153 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:01.479340 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35efd0c2-e1b2-4b71-992e-81fc45525153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:01.505316 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-35efd0c2-e1b2-4b71-992e-81fc45525153 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:16:01.507215 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-35efd0c2-e1b2-4b71-992e-81fc45525153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:16:01.526898 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bacb8e2-ea5d-493f-9641-d0adf5f8da4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:16:01.527796 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bacb8e2-ea5d-493f-9641-d0adf5f8da4f admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:01.528726 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bacb8e2-ea5d-493f-9641-d0adf5f8da4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:01.529013 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bacb8e2-ea5d-493f-9641-d0adf5f8da4f admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:01.529815 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bacb8e2-ea5d-493f-9641-d0adf5f8da4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:16:01.535256 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bacb8e2-ea5d-493f-9641-d0adf5f8da4f admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:16:01.561183 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 924/1862] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:16:01 2022] GET /identity/v3/roles => generated 1420 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:16:01.562349 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bacb8e2-ea5d-493f-9641-d0adf5f8da4f admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:16:01.564405 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-5bacb8e2-ea5d-493f-9641-d0adf5f8da4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:16:01.578473 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:16:01.585774 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bacb8e2-ea5d-493f-9641-d0adf5f8da4f admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:16:01.593500 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aa8a74-87b8-4b8b-8759-574b7e12967a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:16:01.594352 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aa8a74-87b8-4b8b-8759-574b7e12967a admin admin] REQUEST_METHOD: `PUT` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:01.594965 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aa8a74-87b8-4b8b-8759-574b7e12967a admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:01.595420 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aa8a74-87b8-4b8b-8759-574b7e12967a admin admin] PATH_INFO: `/v3/projects/1eb1b878dcf248639e486c467de8378d/users/a5f69414adb942c69a5e48b2cb94ceaf/roles/4eea39e24b06431aa357053fe7907a3d` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:01.622282 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 939/1863] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:16:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:16:01.624050 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92aa8a74-87b8-4b8b-8759-574b7e12967a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1eb1b878dcf248639e486c467de8378d, user_id=a5f69414adb942c69a5e48b2cb94ceaf, role_id=4eea39e24b06431aa357053fe7907a3d)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:16:01.629834 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92aa8a74-87b8-4b8b-8759-574b7e12967a admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:16:01.635515 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d1ac4c-9292-4159-aab5-562734472e4d None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:01.635963 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d1ac4c-9292-4159-aab5-562734472e4d None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:01.636357 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d1ac4c-9292-4159-aab5-562734472e4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:01.672213 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 925/1864] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:16:01 2022] PUT /identity/v3/projects/1eb1b878dcf248639e486c467de8378d/users/a5f69414adb942c69a5e48b2cb94ceaf/roles/4eea39e24b06431aa357053fe7907a3d => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:16:01.686202 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-c4d1ac4c-9292-4159-aab5-562734472e4d None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:16:01.686202 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-c4d1ac4c-9292-4159-aab5-562734472e4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:16:01.691324 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf28dc54-621f-459c-9a4e-86d293e37431 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:01.692533 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf28dc54-621f-459c-9a4e-86d293e37431 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:01.693385 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf28dc54-621f-459c-9a4e-86d293e37431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:01.823723 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 940/1865] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:16:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:16:01.826350 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-cf28dc54-621f-459c-9a4e-86d293e37431 None None] MFA Rules not processed for user `a5f69414adb942c69a5e48b2cb94ceaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:16:01.829252 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d1ac4c-9292-4159-aab5-562734472e4d None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:16:01.835202 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-cf28dc54-621f-459c-9a4e-86d293e37431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:16:01.856412 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d1ac4c-9292-4159-aab5-562734472e4d None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:16:01.867509 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb3b95e-e818-424b-816b-975e0487ab78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:16:01.868830 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb3b95e-e818-424b-816b-975e0487ab78 admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:01.869396 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb3b95e-e818-424b-816b-975e0487ab78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:01.870147 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb3b95e-e818-424b-816b-975e0487ab78 admin admin] PATH_INFO: `/v3/domains` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:01.871508 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb3b95e-e818-424b-816b-975e0487ab78 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:16:01.872131 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb3b95e-e818-424b-816b-975e0487ab78 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:16:01.876367 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb3b95e-e818-424b-816b-975e0487ab78 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:16:01.919242 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 941/1866] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 6 20:16:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:16:01.920822 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb3b95e-e818-424b-816b-975e0487ab78 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:16:01.946345 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb3b95e-e818-424b-816b-975e0487ab78 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:16:01.966259 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 926/1867] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:16:01 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:16:01.966826 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f876dd9-c150-4853-8d0d-e226b159c1e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:16:01.966826 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f876dd9-c150-4853-8d0d-e226b159c1e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:01.966826 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f876dd9-c150-4853-8d0d-e226b159c1e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:01.966826 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f876dd9-c150-4853-8d0d-e226b159c1e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:01.967380 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f876dd9-c150-4853-8d0d-e226b159c1e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:16:01.969921 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f876dd9-c150-4853-8d0d-e226b159c1e3 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:16:01.998635 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf28dc54-621f-459c-9a4e-86d293e37431 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:16:02.000497 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-cf28dc54-621f-459c-9a4e-86d293e37431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:16:02.056761 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 942/1868] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:16:01 2022] POST /identity/v3/projects => generated 371 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:16:02.058766 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f876dd9-c150-4853-8d0d-e226b159c1e3 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:16:02.080878 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:16:02.135470 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf28dc54-621f-459c-9a4e-86d293e37431 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:16:02.162436 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f876dd9-c150-4853-8d0d-e226b159c1e3 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:16:02.168292 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bbfb5f-8494-40a9-bb88-52ef4bd492ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:16:02.168540 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bbfb5f-8494-40a9-bb88-52ef4bd492ce admin admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:02.168873 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bbfb5f-8494-40a9-bb88-52ef4bd492ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:02.169105 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bbfb5f-8494-40a9-bb88-52ef4bd492ce admin admin] PATH_INFO: `/v3/users` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:02.169863 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bbfb5f-8494-40a9-bb88-52ef4bd492ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:16:02.171937 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bbfb5f-8494-40a9-bb88-52ef4bd492ce admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:16:02.208809 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 943/1869] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:16:02 2022] POST /identity/v3/users => generated 286 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 20:16:02.214276 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300b8933-b75a-445c-b3fa-e4504996bd63 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07af18f35ffc42a295278d5349e06381', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:16:02.215087 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bbfb5f-8494-40a9-bb88-52ef4bd492ce admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:16:02.215243 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300b8933-b75a-445c-b3fa-e4504996bd63 service glance] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:02.215457 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300b8933-b75a-445c-b3fa-e4504996bd63 service glance] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:02.215620 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300b8933-b75a-445c-b3fa-e4504996bd63 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:02.242083 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bbfb5f-8494-40a9-bb88-52ef4bd492ce admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:16:02.254049 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e995a0f-6ede-44a9-8e72-7fb3a4c588de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:16:02.255280 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e995a0f-6ede-44a9-8e72-7fb3a4c588de admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:02.255920 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e995a0f-6ede-44a9-8e72-7fb3a4c588de admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:02.256358 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e995a0f-6ede-44a9-8e72-7fb3a4c588de admin admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:02.257690 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e995a0f-6ede-44a9-8e72-7fb3a4c588de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:16:02.260949 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e995a0f-6ede-44a9-8e72-7fb3a4c588de admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:16:02.279184 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 944/1870] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:16:02 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:16:02.285385 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e995a0f-6ede-44a9-8e72-7fb3a4c588de admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:16:02.296980 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e995a0f-6ede-44a9-8e72-7fb3a4c588de admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:16:02.304776 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8373d642-e797-4ac2-b22b-828b93182c88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:16:02.305671 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8373d642-e797-4ac2-b22b-828b93182c88 admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:02.305923 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8373d642-e797-4ac2-b22b-828b93182c88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:02.306269 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8373d642-e797-4ac2-b22b-828b93182c88 admin admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:02.307029 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8373d642-e797-4ac2-b22b-828b93182c88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:16:02.309145 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8373d642-e797-4ac2-b22b-828b93182c88 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:16:02.318530 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300b8933-b75a-445c-b3fa-e4504996bd63 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:16:02.320248 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300b8933-b75a-445c-b3fa-e4504996bd63 service glance] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:16:02.324012 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 945/1871] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:16:02 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:16:02.329593 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8373d642-e797-4ac2-b22b-828b93182c88 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:16:02.333553 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 927/1872] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:16:01 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 20:16:02.342921 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8373d642-e797-4ac2-b22b-828b93182c88 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:16:02.350731 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85d455e-334e-424e-8c08-3268b172fd54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:16:02.351759 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85d455e-334e-424e-8c08-3268b172fd54 admin admin] REQUEST_METHOD: `PUT` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:02.352261 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85d455e-334e-424e-8c08-3268b172fd54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:02.352733 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85d455e-334e-424e-8c08-3268b172fd54 admin admin] PATH_INFO: `/v3/projects/2ca4ba99b8a142d1aaf5ab9df178d981/users/861d88f0f5674b299575ad2e81dbaaf6/roles/4eea39e24b06431aa357053fe7907a3d` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:02.381959 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:16:02.391183 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85d455e-334e-424e-8c08-3268b172fd54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ca4ba99b8a142d1aaf5ab9df178d981, user_id=861d88f0f5674b299575ad2e81dbaaf6, role_id=4eea39e24b06431aa357053fe7907a3d)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:16:02.394864 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85d455e-334e-424e-8c08-3268b172fd54 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:16:02.428240 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 946/1873] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:16:02 2022] PUT /identity/v3/projects/2ca4ba99b8a142d1aaf5ab9df178d981/users/861d88f0f5674b299575ad2e81dbaaf6/roles/4eea39e24b06431aa357053fe7907a3d => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:16:02.431592 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f112ee-6877-4802-b8ef-46daea5739e9 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:02.432438 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f112ee-6877-4802-b8ef-46daea5739e9 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:02.432933 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f112ee-6877-4802-b8ef-46daea5739e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:02.467413 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-11f112ee-6877-4802-b8ef-46daea5739e9 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:16:02.469368 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-11f112ee-6877-4802-b8ef-46daea5739e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:16:02.633704 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 928/1874] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:16:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:16:02.662570 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85d455e-334e-424e-8c08-3268b172fd54 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:16:02.665969 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-b85d455e-334e-424e-8c08-3268b172fd54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:16:02.911116 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85d455e-334e-424e-8c08-3268b172fd54 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:16:02.941649 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2254e84-2fed-4ec2-92ab-d148784e1667 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:16:02.952966 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2254e84-2fed-4ec2-92ab-d148784e1667 service neutron] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:02.954549 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2254e84-2fed-4ec2-92ab-d148784e1667 service neutron] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:02.955336 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2254e84-2fed-4ec2-92ab-d148784e1667 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:02.985469 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2254e84-2fed-4ec2-92ab-d148784e1667 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:16:02.990925 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2254e84-2fed-4ec2-92ab-d148784e1667 service neutron] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:16:03.039860 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 947/1875] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:16:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:16:05.168447 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2993b3a7-b7a7-4d91-8a12-526920662460 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:05.171084 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2993b3a7-b7a7-4d91-8a12-526920662460 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:05.172230 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2993b3a7-b7a7-4d91-8a12-526920662460 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:05.244895 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-2993b3a7-b7a7-4d91-8a12-526920662460 None None] MFA Rules not processed for user `861d88f0f5674b299575ad2e81dbaaf6`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:16:05.246285 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-2993b3a7-b7a7-4d91-8a12-526920662460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:16:05.336107 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:16:05.440336 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 929/1876] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:16:05 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:16:05.462162 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2254e84-2fed-4ec2-92ab-d148784e1667 service neutron] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:16:05.464435 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-a2254e84-2fed-4ec2-92ab-d148784e1667 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:16:05.680616 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2254e84-2fed-4ec2-92ab-d148784e1667 service neutron] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:16:05.690328 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef563548-453e-40c0-bbea-e3c8daed2c5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df1bca993bc142908cc7cf85eabd2b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:16:05.691956 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef563548-453e-40c0-bbea-e3c8daed2c5a service cinder] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:05.692840 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef563548-453e-40c0-bbea-e3c8daed2c5a service cinder] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:05.693646 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef563548-453e-40c0-bbea-e3c8daed2c5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:05.709542 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef563548-453e-40c0-bbea-e3c8daed2c5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:16:05.713214 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef563548-453e-40c0-bbea-e3c8daed2c5a service cinder] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:16:05.735980 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 948/1877] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:16:05 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:16:09.570021 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdad4251-ed59-497f-8c23-4788d204f94a None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:09.570875 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdad4251-ed59-497f-8c23-4788d204f94a None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:09.571028 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdad4251-ed59-497f-8c23-4788d204f94a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:09.622290 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-fdad4251-ed59-497f-8c23-4788d204f94a None None] MFA Rules not processed for user `861d88f0f5674b299575ad2e81dbaaf6`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:16:09.624547 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-fdad4251-ed59-497f-8c23-4788d204f94a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:16:09.650911 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 930/1878] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:16:09 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:16:15.727356 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=55,keystone:INSERT=1 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:16:19.605083 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=19 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:16:26.798556 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef563548-453e-40c0-bbea-e3c8daed2c5a service cinder] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:16:26.801904 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-ef563548-453e-40c0-bbea-e3c8daed2c5a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:16:26.848209 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef563548-453e-40c0-bbea-e3c8daed2c5a service cinder] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:16:26.863421 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb801201-672a-443e-b792-5ffb259f6c1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:16:26.865027 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb801201-672a-443e-b792-5ffb259f6c1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:26.865869 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb801201-672a-443e-b792-5ffb259f6c1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:26.866780 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb801201-672a-443e-b792-5ffb259f6c1d admin admin] PATH_INFO: `/v3/users/390fc4ffdaf24a7d887c10e4a7a8928b` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:26.873064 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb801201-672a-443e-b792-5ffb259f6c1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=390fc4ffdaf24a7d887c10e4a7a8928b)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:16:26.881728 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb801201-672a-443e-b792-5ffb259f6c1d admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:16:27.034883 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-eb801201-672a-443e-b792-5ffb259f6c1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '390fc4ffdaf24a7d887c10e4a7a8928b'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:16:27.037543 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-eb801201-672a-443e-b792-5ffb259f6c1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '390fc4ffdaf24a7d887c10e4a7a8928b'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:16:27.066788 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-eb801201-672a-443e-b792-5ffb259f6c1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '390fc4ffdaf24a7d887c10e4a7a8928b'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:16:27.067592 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-eb801201-672a-443e-b792-5ffb259f6c1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '390fc4ffdaf24a7d887c10e4a7a8928b'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:16:27.081027 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-eb801201-672a-443e-b792-5ffb259f6c1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '390fc4ffdaf24a7d887c10e4a7a8928b'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:16:27.104529 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 949/1879] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:16:26 2022] DELETE /identity/v3/users/390fc4ffdaf24a7d887c10e4a7a8928b => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:16:27.110541 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4ca0c8-8430-4d16-87d4-94225c32ccbd None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:27.111283 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4ca0c8-8430-4d16-87d4-94225c32ccbd None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:27.112525 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4ca0c8-8430-4d16-87d4-94225c32ccbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:27.143261 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-dd4ca0c8-8430-4d16-87d4-94225c32ccbd None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:16:27.149471 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-dd4ca0c8-8430-4d16-87d4-94225c32ccbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:16:27.323051 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 931/1880] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:16:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:16:27.339121 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb801201-672a-443e-b792-5ffb259f6c1d admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:16:27.341488 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-eb801201-672a-443e-b792-5ffb259f6c1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:16:27.623337 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb801201-672a-443e-b792-5ffb259f6c1d admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:16:27.633758 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bebeb6-5df9-4ca8-87ae-19a0eee5c73f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:16:27.635047 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bebeb6-5df9-4ca8-87ae-19a0eee5c73f service neutron] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:27.635656 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bebeb6-5df9-4ca8-87ae-19a0eee5c73f service neutron] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:27.636173 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bebeb6-5df9-4ca8-87ae-19a0eee5c73f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:27.679447 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bebeb6-5df9-4ca8-87ae-19a0eee5c73f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:16:27.684848 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bebeb6-5df9-4ca8-87ae-19a0eee5c73f service neutron] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:16:27.717848 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 950/1881] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:16:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:16:28.952577 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4ca0c8-8430-4d16-87d4-94225c32ccbd None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:16:28.955564 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-dd4ca0c8-8430-4d16-87d4-94225c32ccbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:16:28.998225 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4ca0c8-8430-4d16-87d4-94225c32ccbd None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:16:29.015266 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa36ed77-dacd-4fb2-b1ee-b11c62a2290f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:16:29.016468 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa36ed77-dacd-4fb2-b1ee-b11c62a2290f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:29.016785 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa36ed77-dacd-4fb2-b1ee-b11c62a2290f admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:29.017262 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa36ed77-dacd-4fb2-b1ee-b11c62a2290f admin admin] PATH_INFO: `/v3/projects/95cb7f1253d644cba5e3309588410579` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:29.019934 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa36ed77-dacd-4fb2-b1ee-b11c62a2290f admin admin] RBAC: Authorizing `identity:delete_project(project_id=95cb7f1253d644cba5e3309588410579)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:16:29.032449 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa36ed77-dacd-4fb2-b1ee-b11c62a2290f admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:16:29.228331 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-aa36ed77-dacd-4fb2-b1ee-b11c62a2290f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95cb7f1253d644cba5e3309588410579', 'request_id': 'req-aa36ed77-dacd-4fb2-b1ee-b11c62a2290f'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:16:29.239838 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-aa36ed77-dacd-4fb2-b1ee-b11c62a2290f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95cb7f1253d644cba5e3309588410579', 'request_id': 'req-aa36ed77-dacd-4fb2-b1ee-b11c62a2290f'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:16:29.273173 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-aa36ed77-dacd-4fb2-b1ee-b11c62a2290f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95cb7f1253d644cba5e3309588410579', 'request_id': 'req-aa36ed77-dacd-4fb2-b1ee-b11c62a2290f'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:16:29.347597 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-aa36ed77-dacd-4fb2-b1ee-b11c62a2290f admin admin] The token cache is being invalidate because project 95cb7f1253d644cba5e3309588410579 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81771) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:16:29.348149 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-aa36ed77-dacd-4fb2-b1ee-b11c62a2290f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:16:29.349556 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-aa36ed77-dacd-4fb2-b1ee-b11c62a2290f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:16:29.353565 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 932/1882] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:16:28 2022] DELETE /identity/v3/projects/95cb7f1253d644cba5e3309588410579 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:16:29.360200 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bebeb6-5df9-4ca8-87ae-19a0eee5c73f service neutron] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:16:29.392501 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bebeb6-5df9-4ca8-87ae-19a0eee5c73f service neutron] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:16:29.679539 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a867422-92d0-4178-b5d7-772368bc7c3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:16:29.680630 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a867422-92d0-4178-b5d7-772368bc7c3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:29.681773 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a867422-92d0-4178-b5d7-772368bc7c3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:29.681773 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a867422-92d0-4178-b5d7-772368bc7c3c admin admin] PATH_INFO: `/v3/users/1f61a795f30a420c83aa65cebab57c60` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:29.692095 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a867422-92d0-4178-b5d7-772368bc7c3c admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f61a795f30a420c83aa65cebab57c60)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:16:29.695914 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a867422-92d0-4178-b5d7-772368bc7c3c admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:16:29.730930 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:16:29.906269 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-5a867422-92d0-4178-b5d7-772368bc7c3c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f61a795f30a420c83aa65cebab57c60'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:16:29.907459 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-5a867422-92d0-4178-b5d7-772368bc7c3c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f61a795f30a420c83aa65cebab57c60'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:16:29.944553 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-5a867422-92d0-4178-b5d7-772368bc7c3c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f61a795f30a420c83aa65cebab57c60'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:16:29.945412 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-5a867422-92d0-4178-b5d7-772368bc7c3c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f61a795f30a420c83aa65cebab57c60'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:16:29.960082 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-5a867422-92d0-4178-b5d7-772368bc7c3c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f61a795f30a420c83aa65cebab57c60'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:16:29.990915 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 951/1883] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:16:29 2022] DELETE /identity/v3/users/1f61a795f30a420c83aa65cebab57c60 => generated 0 bytes in 632 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:16:30.900389 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa36ed77-dacd-4fb2-b1ee-b11c62a2290f admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:16:30.903496 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-aa36ed77-dacd-4fb2-b1ee-b11c62a2290f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:16:30.954126 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa36ed77-dacd-4fb2-b1ee-b11c62a2290f admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:16:31.168131 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e485118e-7763-4525-8702-0182479c3457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:16:31.169101 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e485118e-7763-4525-8702-0182479c3457 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:31.169657 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e485118e-7763-4525-8702-0182479c3457 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:31.170185 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e485118e-7763-4525-8702-0182479c3457 admin admin] PATH_INFO: `/v3/projects/0912369993c943529c87b2f8d6e02d1e` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:31.174901 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e485118e-7763-4525-8702-0182479c3457 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0912369993c943529c87b2f8d6e02d1e)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:16:31.179000 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e485118e-7763-4525-8702-0182479c3457 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:16:31.249383 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:16:31.337280 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-e485118e-7763-4525-8702-0182479c3457 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0912369993c943529c87b2f8d6e02d1e', 'request_id': 'req-e485118e-7763-4525-8702-0182479c3457'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:16:31.364825 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-e485118e-7763-4525-8702-0182479c3457 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0912369993c943529c87b2f8d6e02d1e', 'request_id': 'req-e485118e-7763-4525-8702-0182479c3457'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:16:31.388659 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-e485118e-7763-4525-8702-0182479c3457 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0912369993c943529c87b2f8d6e02d1e', 'request_id': 'req-e485118e-7763-4525-8702-0182479c3457'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:16:31.417986 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-e485118e-7763-4525-8702-0182479c3457 admin admin] The token cache is being invalidate because project 0912369993c943529c87b2f8d6e02d1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81771) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:16:31.418304 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-e485118e-7763-4525-8702-0182479c3457 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:16:31.419432 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-e485118e-7763-4525-8702-0182479c3457 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:16:31.424412 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 933/1884] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:16:30 2022] DELETE /identity/v3/projects/0912369993c943529c87b2f8d6e02d1e => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:16:31.432516 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d783b9f-c934-4bd8-ab6a-3eb7aa4cd0a5 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:31.433328 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d783b9f-c934-4bd8-ab6a-3eb7aa4cd0a5 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:31.434167 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d783b9f-c934-4bd8-ab6a-3eb7aa4cd0a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:31.488164 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-5d783b9f-c934-4bd8-ab6a-3eb7aa4cd0a5 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:16:31.490561 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-5d783b9f-c934-4bd8-ab6a-3eb7aa4cd0a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:16:31.610635 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 952/1885] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:16:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:16:31.629246 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d301b3f4-c744-4ad3-9b31-671df5888fe9 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:31.629246 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d301b3f4-c744-4ad3-9b31-671df5888fe9 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:31.629246 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d301b3f4-c744-4ad3-9b31-671df5888fe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:31.666983 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-d301b3f4-c744-4ad3-9b31-671df5888fe9 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:16:31.668349 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-d301b3f4-c744-4ad3-9b31-671df5888fe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:16:31.682296 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 934/1886] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:16:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:16:31.688260 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d783b9f-c934-4bd8-ab6a-3eb7aa4cd0a5 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:16:31.718184 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d783b9f-c934-4bd8-ab6a-3eb7aa4cd0a5 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:16:31.726945 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59feb92a-c98b-49ad-b3bc-4c43ca76b301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:16:31.727910 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59feb92a-c98b-49ad-b3bc-4c43ca76b301 admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:31.728751 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59feb92a-c98b-49ad-b3bc-4c43ca76b301 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:31.729246 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59feb92a-c98b-49ad-b3bc-4c43ca76b301 admin admin] PATH_INFO: `/v3/domains` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:31.730633 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59feb92a-c98b-49ad-b3bc-4c43ca76b301 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:16:31.731140 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59feb92a-c98b-49ad-b3bc-4c43ca76b301 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:16:31.734487 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59feb92a-c98b-49ad-b3bc-4c43ca76b301 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:16:31.768134 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 953/1887] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 6 20:16:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:16:31.773628 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d301b3f4-c744-4ad3-9b31-671df5888fe9 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:16:31.786286 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d301b3f4-c744-4ad3-9b31-671df5888fe9 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:16:31.794830 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc6a6c6-4d91-40b1-aab6-5a3ae8e51071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:16:31.795732 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc6a6c6-4d91-40b1-aab6-5a3ae8e51071 admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:31.796111 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc6a6c6-4d91-40b1-aab6-5a3ae8e51071 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:31.796451 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc6a6c6-4d91-40b1-aab6-5a3ae8e51071 admin admin] PATH_INFO: `/v3/projects` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:31.797275 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc6a6c6-4d91-40b1-aab6-5a3ae8e51071 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:16:31.800717 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc6a6c6-4d91-40b1-aab6-5a3ae8e51071 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:16:31.847899 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 935/1888] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:16:31 2022] POST /identity/v3/projects => generated 383 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 20:16:31.853192 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59feb92a-c98b-49ad-b3bc-4c43ca76b301 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:16:31.869969 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59feb92a-c98b-49ad-b3bc-4c43ca76b301 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:16:31.982331 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000a2b5e-91db-465b-8688-7a910a22b047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:16:31.982723 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000a2b5e-91db-465b-8688-7a910a22b047 admin admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:31.983010 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000a2b5e-91db-465b-8688-7a910a22b047 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:31.983661 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000a2b5e-91db-465b-8688-7a910a22b047 admin admin] PATH_INFO: `/v3/users` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:31.984484 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000a2b5e-91db-465b-8688-7a910a22b047 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:16:31.990096 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000a2b5e-91db-465b-8688-7a910a22b047 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:16:32.026339 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 954/1889] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:16:31 2022] POST /identity/v3/users => generated 292 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:16:32.037475 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc6a6c6-4d91-40b1-aab6-5a3ae8e51071 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:16:32.060568 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc6a6c6-4d91-40b1-aab6-5a3ae8e51071 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:16:32.069293 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a506d166-776d-4e46-a5ed-19f37991aab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:16:32.070463 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a506d166-776d-4e46-a5ed-19f37991aab0 admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:32.070956 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a506d166-776d-4e46-a5ed-19f37991aab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:32.071404 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a506d166-776d-4e46-a5ed-19f37991aab0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:32.072593 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a506d166-776d-4e46-a5ed-19f37991aab0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:16:32.076368 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a506d166-776d-4e46-a5ed-19f37991aab0 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:16:32.090769 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 936/1890] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:16:32 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:16:32.097080 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000a2b5e-91db-465b-8688-7a910a22b047 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:16:32.115449 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000a2b5e-91db-465b-8688-7a910a22b047 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:16:32.122810 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebda7bd0-14f7-43a5-a06a-2d58d426f017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:16:32.124485 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebda7bd0-14f7-43a5-a06a-2d58d426f017 admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:32.124942 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebda7bd0-14f7-43a5-a06a-2d58d426f017 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:32.125482 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebda7bd0-14f7-43a5-a06a-2d58d426f017 admin admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:32.127228 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebda7bd0-14f7-43a5-a06a-2d58d426f017 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:16:32.130645 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebda7bd0-14f7-43a5-a06a-2d58d426f017 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:16:32.141039 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:16:32.152097 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 955/1891] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:16:32 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:16:32.159276 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a506d166-776d-4e46-a5ed-19f37991aab0 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:16:32.174039 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a506d166-776d-4e46-a5ed-19f37991aab0 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:16:32.181825 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99530d4a-1173-46b8-bb94-f4bf2dd02332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:16:32.183001 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99530d4a-1173-46b8-bb94-f4bf2dd02332 admin admin] REQUEST_METHOD: `PUT` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:32.183498 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99530d4a-1173-46b8-bb94-f4bf2dd02332 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:32.184194 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99530d4a-1173-46b8-bb94-f4bf2dd02332 admin admin] PATH_INFO: `/v3/projects/41c783020c004ad2b95c35d207d6d184/users/f5ad5b4cadb54e97abe4603b5d9351d5/roles/4eea39e24b06431aa357053fe7907a3d` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:32.209380 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99530d4a-1173-46b8-bb94-f4bf2dd02332 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41c783020c004ad2b95c35d207d6d184, user_id=f5ad5b4cadb54e97abe4603b5d9351d5, role_id=4eea39e24b06431aa357053fe7907a3d)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:16:32.212365 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99530d4a-1173-46b8-bb94-f4bf2dd02332 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:16:32.231051 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 937/1892] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:16:32 2022] PUT /identity/v3/projects/41c783020c004ad2b95c35d207d6d184/users/f5ad5b4cadb54e97abe4603b5d9351d5/roles/4eea39e24b06431aa357053fe7907a3d => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 20:16:32.244064 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4c6508-1b87-46eb-95cf-81f0a53adfce None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:32.244350 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4c6508-1b87-46eb-95cf-81f0a53adfce None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:32.244698 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4c6508-1b87-46eb-95cf-81f0a53adfce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:32.292688 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-6f4c6508-1b87-46eb-95cf-81f0a53adfce None None] MFA Rules not processed for user `f5ad5b4cadb54e97abe4603b5d9351d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:16:32.294329 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-6f4c6508-1b87-46eb-95cf-81f0a53adfce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:16:32.376620 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 956/1893] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:16:32 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:16:32.383307 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99530d4a-1173-46b8-bb94-f4bf2dd02332 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:16:32.468542 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99530d4a-1173-46b8-bb94-f4bf2dd02332 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:16:32.477248 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18ce4ab-3a43-4825-88ab-69646286cb31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:16:32.478245 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18ce4ab-3a43-4825-88ab-69646286cb31 admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:32.478731 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18ce4ab-3a43-4825-88ab-69646286cb31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:32.479189 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18ce4ab-3a43-4825-88ab-69646286cb31 admin admin] PATH_INFO: `/v3/projects` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:32.480327 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18ce4ab-3a43-4825-88ab-69646286cb31 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:16:32.484071 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18ce4ab-3a43-4825-88ab-69646286cb31 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:16:32.538000 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 938/1894] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:16:32 2022] POST /identity/v3/projects => generated 383 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 20:16:32.543010 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4c6508-1b87-46eb-95cf-81f0a53adfce None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:16:32.566284 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4c6508-1b87-46eb-95cf-81f0a53adfce None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:16:32.681429 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213ca51e-a908-492f-aa25-7da3851fc200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:16:32.682117 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213ca51e-a908-492f-aa25-7da3851fc200 admin admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:32.682404 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213ca51e-a908-492f-aa25-7da3851fc200 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:32.682726 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213ca51e-a908-492f-aa25-7da3851fc200 admin admin] PATH_INFO: `/v3/users` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:32.683525 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213ca51e-a908-492f-aa25-7da3851fc200 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:16:32.685880 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213ca51e-a908-492f-aa25-7da3851fc200 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:16:32.719124 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 957/1895] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:16:32 2022] POST /identity/v3/users => generated 298 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 20:16:32.720860 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18ce4ab-3a43-4825-88ab-69646286cb31 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:16:32.737753 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18ce4ab-3a43-4825-88ab-69646286cb31 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:16:32.752859 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc3fdbd-6222-4727-9379-3bc1273a694a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:16:32.755673 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc3fdbd-6222-4727-9379-3bc1273a694a admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:32.756414 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc3fdbd-6222-4727-9379-3bc1273a694a admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:32.756733 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc3fdbd-6222-4727-9379-3bc1273a694a admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:32.758116 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc3fdbd-6222-4727-9379-3bc1273a694a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:16:32.764575 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc3fdbd-6222-4727-9379-3bc1273a694a admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:16:32.799971 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 939/1896] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:16:32 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 20:16:32.801290 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213ca51e-a908-492f-aa25-7da3851fc200 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:16:32.815563 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213ca51e-a908-492f-aa25-7da3851fc200 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:16:32.821542 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12706e2b-10fa-4011-8933-23c6ef295fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:16:32.822385 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12706e2b-10fa-4011-8933-23c6ef295fc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:32.822739 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12706e2b-10fa-4011-8933-23c6ef295fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:32.823253 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12706e2b-10fa-4011-8933-23c6ef295fc4 admin admin] PATH_INFO: `/v3/projects/71a4178841ca425ca274fc88e1dec135/users/63ebc0b53cdb4bb691ea214b8a58585e/roles/e0ab41d8ae22474688b5361bcc75e0f0` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:32.841890 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12706e2b-10fa-4011-8933-23c6ef295fc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71a4178841ca425ca274fc88e1dec135, user_id=63ebc0b53cdb4bb691ea214b8a58585e, role_id=e0ab41d8ae22474688b5361bcc75e0f0)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:16:32.844927 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12706e2b-10fa-4011-8933-23c6ef295fc4 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:16:32.873890 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc3fdbd-6222-4727-9379-3bc1273a694a admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:16:32.874528 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 958/1897] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:16:32 2022] PUT /identity/v3/projects/71a4178841ca425ca274fc88e1dec135/users/63ebc0b53cdb4bb691ea214b8a58585e/roles/e0ab41d8ae22474688b5361bcc75e0f0 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 20:16:32.889805 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc3fdbd-6222-4727-9379-3bc1273a694a admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:16:32.926768 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:16:33.038023 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e379d6-36fc-415d-a773-86e2748298eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:16:33.038712 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e379d6-36fc-415d-a773-86e2748298eb admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:33.038972 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e379d6-36fc-415d-a773-86e2748298eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:33.039256 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e379d6-36fc-415d-a773-86e2748298eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:33.039919 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e379d6-36fc-415d-a773-86e2748298eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:16:33.041836 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e379d6-36fc-415d-a773-86e2748298eb admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:16:33.060799 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12706e2b-10fa-4011-8933-23c6ef295fc4 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:16:33.067546 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 940/1898] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:16:32 2022] GET /identity/v3/roles => generated 1420 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:16:33.076083 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12706e2b-10fa-4011-8933-23c6ef295fc4 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:16:33.087722 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f39f70-2e66-480f-bc24-8f169eca7f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:16:33.088441 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f39f70-2e66-480f-bc24-8f169eca7f26 admin admin] REQUEST_METHOD: `PUT` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:33.088835 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f39f70-2e66-480f-bc24-8f169eca7f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:33.089118 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f39f70-2e66-480f-bc24-8f169eca7f26 admin admin] PATH_INFO: `/v3/projects/71a4178841ca425ca274fc88e1dec135/users/63ebc0b53cdb4bb691ea214b8a58585e/roles/e0ab41d8ae22474688b5361bcc75e0f0` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:33.092821 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f39f70-2e66-480f-bc24-8f169eca7f26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71a4178841ca425ca274fc88e1dec135, user_id=63ebc0b53cdb4bb691ea214b8a58585e, role_id=e0ab41d8ae22474688b5361bcc75e0f0)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:16:33.098353 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f39f70-2e66-480f-bc24-8f169eca7f26 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:16:33.117284 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 959/1899] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:16:33 2022] PUT /identity/v3/projects/71a4178841ca425ca274fc88e1dec135/users/63ebc0b53cdb4bb691ea214b8a58585e/roles/e0ab41d8ae22474688b5361bcc75e0f0 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:16:33.130682 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e66f17b-0bc4-4cf8-b3b2-a713636713ad None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:33.130948 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e66f17b-0bc4-4cf8-b3b2-a713636713ad None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:33.131333 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e66f17b-0bc4-4cf8-b3b2-a713636713ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:33.179537 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-5e66f17b-0bc4-4cf8-b3b2-a713636713ad None None] MFA Rules not processed for user `63ebc0b53cdb4bb691ea214b8a58585e`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:16:33.181989 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-5e66f17b-0bc4-4cf8-b3b2-a713636713ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:16:33.283051 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 941/1900] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:16:33 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:16:33.299608 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f39f70-2e66-480f-bc24-8f169eca7f26 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:16:33.302418 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-85f39f70-2e66-480f-bc24-8f169eca7f26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:16:33.392899 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:16:33.417135 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f39f70-2e66-480f-bc24-8f169eca7f26 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:16:33.422674 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de654cf-8002-44f3-8445-fbb48b254321 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df1bca993bc142908cc7cf85eabd2b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:16:33.425490 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de654cf-8002-44f3-8445-fbb48b254321 service cinder] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:33.426134 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de654cf-8002-44f3-8445-fbb48b254321 service cinder] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:33.426657 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de654cf-8002-44f3-8445-fbb48b254321 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:33.443206 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de654cf-8002-44f3-8445-fbb48b254321 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:16:33.445690 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de654cf-8002-44f3-8445-fbb48b254321 service cinder] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:16:33.463344 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 960/1901] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:16:33 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:16:33.633866 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e66f17b-0bc4-4cf8-b3b2-a713636713ad None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:16:33.652401 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e66f17b-0bc4-4cf8-b3b2-a713636713ad None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:16:33.661112 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee916d6-c1b5-4f97-81d6-9fb40f93416e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df1bca993bc142908cc7cf85eabd2b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:16:33.662315 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee916d6-c1b5-4f97-81d6-9fb40f93416e service cinder] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:33.662830 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee916d6-c1b5-4f97-81d6-9fb40f93416e service cinder] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:33.663335 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee916d6-c1b5-4f97-81d6-9fb40f93416e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:33.786215 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee916d6-c1b5-4f97-81d6-9fb40f93416e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:16:33.789058 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee916d6-c1b5-4f97-81d6-9fb40f93416e service cinder] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:16:33.811297 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 942/1902] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:16:33 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:16:36.938287 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de654cf-8002-44f3-8445-fbb48b254321 service cinder] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:16:37.077280 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de654cf-8002-44f3-8445-fbb48b254321 service cinder] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:16:37.087303 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f445b9bc-d1c1-4ddf-afac-b542d41133b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:16:37.088377 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f445b9bc-d1c1-4ddf-afac-b542d41133b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:37.088876 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f445b9bc-d1c1-4ddf-afac-b542d41133b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:37.089367 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f445b9bc-d1c1-4ddf-afac-b542d41133b3 admin admin] PATH_INFO: `/v3/users/f5ad5b4cadb54e97abe4603b5d9351d5` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:37.091646 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f445b9bc-d1c1-4ddf-afac-b542d41133b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5ad5b4cadb54e97abe4603b5d9351d5)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:16:37.096980 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f445b9bc-d1c1-4ddf-afac-b542d41133b3 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:16:37.207412 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-f445b9bc-d1c1-4ddf-afac-b542d41133b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5ad5b4cadb54e97abe4603b5d9351d5'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:16:37.208366 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-f445b9bc-d1c1-4ddf-afac-b542d41133b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5ad5b4cadb54e97abe4603b5d9351d5'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:16:37.229788 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-f445b9bc-d1c1-4ddf-afac-b542d41133b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5ad5b4cadb54e97abe4603b5d9351d5'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:16:37.230850 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-f445b9bc-d1c1-4ddf-afac-b542d41133b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5ad5b4cadb54e97abe4603b5d9351d5'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:16:37.249120 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-f445b9bc-d1c1-4ddf-afac-b542d41133b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5ad5b4cadb54e97abe4603b5d9351d5'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:16:37.267908 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 961/1903] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:16:36 2022] DELETE /identity/v3/users/f5ad5b4cadb54e97abe4603b5d9351d5 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:16:37.273595 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f282fda0-b445-4367-a718-143fd625e3d0 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:37.274446 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f282fda0-b445-4367-a718-143fd625e3d0 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:37.274446 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f282fda0-b445-4367-a718-143fd625e3d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:37.298006 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-f282fda0-b445-4367-a718-143fd625e3d0 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:16:37.299999 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-f282fda0-b445-4367-a718-143fd625e3d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:16:37.377847 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:16:37.417673 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 943/1904] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:16:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:16:37.428201 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f445b9bc-d1c1-4ddf-afac-b542d41133b3 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:16:37.430592 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-f445b9bc-d1c1-4ddf-afac-b542d41133b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:16:37.506848 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f445b9bc-d1c1-4ddf-afac-b542d41133b3 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:16:37.510925 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f671e7cc-94a3-400f-bbe1-839ab066eeee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:16:37.511525 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f671e7cc-94a3-400f-bbe1-839ab066eeee service neutron] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:37.511793 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f671e7cc-94a3-400f-bbe1-839ab066eeee service neutron] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:37.512073 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f671e7cc-94a3-400f-bbe1-839ab066eeee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:37.523234 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f671e7cc-94a3-400f-bbe1-839ab066eeee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:16:37.524987 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f671e7cc-94a3-400f-bbe1-839ab066eeee service neutron] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:16:37.537440 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 962/1905] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:16:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 20:16:37.852808 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f282fda0-b445-4367-a718-143fd625e3d0 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:16:37.854657 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-f282fda0-b445-4367-a718-143fd625e3d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:16:37.872194 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f282fda0-b445-4367-a718-143fd625e3d0 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:16:37.879276 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cf93e0-2fa6-4db1-8487-5a94976f0393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:16:37.880349 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cf93e0-2fa6-4db1-8487-5a94976f0393 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:37.880876 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cf93e0-2fa6-4db1-8487-5a94976f0393 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:37.881396 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cf93e0-2fa6-4db1-8487-5a94976f0393 admin admin] PATH_INFO: `/v3/projects/41c783020c004ad2b95c35d207d6d184` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:37.883512 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cf93e0-2fa6-4db1-8487-5a94976f0393 admin admin] RBAC: Authorizing `identity:delete_project(project_id=41c783020c004ad2b95c35d207d6d184)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:16:37.886117 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cf93e0-2fa6-4db1-8487-5a94976f0393 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:16:37.959225 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-00cf93e0-2fa6-4db1-8487-5a94976f0393 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41c783020c004ad2b95c35d207d6d184', 'request_id': 'req-00cf93e0-2fa6-4db1-8487-5a94976f0393'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:16:37.966954 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-00cf93e0-2fa6-4db1-8487-5a94976f0393 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41c783020c004ad2b95c35d207d6d184', 'request_id': 'req-00cf93e0-2fa6-4db1-8487-5a94976f0393'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:16:37.976678 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-00cf93e0-2fa6-4db1-8487-5a94976f0393 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41c783020c004ad2b95c35d207d6d184', 'request_id': 'req-00cf93e0-2fa6-4db1-8487-5a94976f0393'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:16:37.990223 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-00cf93e0-2fa6-4db1-8487-5a94976f0393 admin admin] The token cache is being invalidate because project 41c783020c004ad2b95c35d207d6d184 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81771) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:16:37.990529 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-00cf93e0-2fa6-4db1-8487-5a94976f0393 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:16:37.991195 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-00cf93e0-2fa6-4db1-8487-5a94976f0393 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:16:37.992749 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 944/1906] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:16:37 2022] DELETE /identity/v3/projects/41c783020c004ad2b95c35d207d6d184 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:16:37.998516 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f671e7cc-94a3-400f-bbe1-839ab066eeee service neutron] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:16:38.010636 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f671e7cc-94a3-400f-bbe1-839ab066eeee service neutron] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:16:38.015163 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:16:38.110232 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1f34aa-d8e2-43d9-a06a-90588c2f30d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:16:38.110998 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1f34aa-d8e2-43d9-a06a-90588c2f30d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:38.111290 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1f34aa-d8e2-43d9-a06a-90588c2f30d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:38.111571 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1f34aa-d8e2-43d9-a06a-90588c2f30d4 admin admin] PATH_INFO: `/v3/users/63ebc0b53cdb4bb691ea214b8a58585e` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:38.113234 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1f34aa-d8e2-43d9-a06a-90588c2f30d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=63ebc0b53cdb4bb691ea214b8a58585e)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:16:38.115361 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1f34aa-d8e2-43d9-a06a-90588c2f30d4 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:16:38.184296 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-ad1f34aa-d8e2-43d9-a06a-90588c2f30d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63ebc0b53cdb4bb691ea214b8a58585e'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:16:38.185224 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-ad1f34aa-d8e2-43d9-a06a-90588c2f30d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63ebc0b53cdb4bb691ea214b8a58585e'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:16:38.220142 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-ad1f34aa-d8e2-43d9-a06a-90588c2f30d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63ebc0b53cdb4bb691ea214b8a58585e'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:16:38.221465 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-ad1f34aa-d8e2-43d9-a06a-90588c2f30d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63ebc0b53cdb4bb691ea214b8a58585e'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:16:38.235381 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-ad1f34aa-d8e2-43d9-a06a-90588c2f30d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63ebc0b53cdb4bb691ea214b8a58585e'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:16:38.250941 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 963/1907] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:16:37 2022] DELETE /identity/v3/users/63ebc0b53cdb4bb691ea214b8a58585e => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:16:38.677045 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cf93e0-2fa6-4db1-8487-5a94976f0393 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:16:38.678788 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-00cf93e0-2fa6-4db1-8487-5a94976f0393 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:16:38.694092 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cf93e0-2fa6-4db1-8487-5a94976f0393 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:16:38.773595 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f9e187-1821-43b6-b53b-c7d7e5e1bd70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:16:38.774297 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f9e187-1821-43b6-b53b-c7d7e5e1bd70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:38.774666 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f9e187-1821-43b6-b53b-c7d7e5e1bd70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:38.774958 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f9e187-1821-43b6-b53b-c7d7e5e1bd70 admin admin] PATH_INFO: `/v3/projects/71a4178841ca425ca274fc88e1dec135` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:38.776388 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f9e187-1821-43b6-b53b-c7d7e5e1bd70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71a4178841ca425ca274fc88e1dec135)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:16:38.778465 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f9e187-1821-43b6-b53b-c7d7e5e1bd70 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:16:38.839994 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-14f9e187-1821-43b6-b53b-c7d7e5e1bd70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71a4178841ca425ca274fc88e1dec135', 'request_id': 'req-14f9e187-1821-43b6-b53b-c7d7e5e1bd70'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:16:38.848658 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-14f9e187-1821-43b6-b53b-c7d7e5e1bd70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71a4178841ca425ca274fc88e1dec135', 'request_id': 'req-14f9e187-1821-43b6-b53b-c7d7e5e1bd70'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:16:38.859257 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-14f9e187-1821-43b6-b53b-c7d7e5e1bd70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71a4178841ca425ca274fc88e1dec135', 'request_id': 'req-14f9e187-1821-43b6-b53b-c7d7e5e1bd70'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:16:38.868616 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:16:38.875888 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-14f9e187-1821-43b6-b53b-c7d7e5e1bd70 admin admin] The token cache is being invalidate because project 71a4178841ca425ca274fc88e1dec135 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81771) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:16:38.877835 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-14f9e187-1821-43b6-b53b-c7d7e5e1bd70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:16:38.879364 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-14f9e187-1821-43b6-b53b-c7d7e5e1bd70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:16:38.882143 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 945/1908] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:16:38 2022] DELETE /identity/v3/projects/71a4178841ca425ca274fc88e1dec135 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:16:38.890904 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc194fe3-989e-4f47-852b-2f1c1a8453c6 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:38.891416 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc194fe3-989e-4f47-852b-2f1c1a8453c6 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:38.892028 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc194fe3-989e-4f47-852b-2f1c1a8453c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:38.926662 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-cc194fe3-989e-4f47-852b-2f1c1a8453c6 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:16:38.927793 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-cc194fe3-989e-4f47-852b-2f1c1a8453c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:16:39.025914 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 964/1909] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:16:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:16:39.030253 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ff6c63-cfe2-4b5d-bd28-0b07852f8a7c None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:39.030733 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ff6c63-cfe2-4b5d-bd28-0b07852f8a7c None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:39.032307 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ff6c63-cfe2-4b5d-bd28-0b07852f8a7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:39.073272 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-60ff6c63-cfe2-4b5d-bd28-0b07852f8a7c None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:16:39.075483 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-60ff6c63-cfe2-4b5d-bd28-0b07852f8a7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:16:39.090589 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 946/1910] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:16:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:16:39.095852 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc194fe3-989e-4f47-852b-2f1c1a8453c6 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:16:39.106999 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc194fe3-989e-4f47-852b-2f1c1a8453c6 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:16:39.112627 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533b6212-62d4-4f20-81e7-6462cfed7aa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:16:39.113540 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533b6212-62d4-4f20-81e7-6462cfed7aa6 admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:39.113860 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533b6212-62d4-4f20-81e7-6462cfed7aa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:39.114138 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533b6212-62d4-4f20-81e7-6462cfed7aa6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:39.114833 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533b6212-62d4-4f20-81e7-6462cfed7aa6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:16:39.115137 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533b6212-62d4-4f20-81e7-6462cfed7aa6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:16:39.116825 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533b6212-62d4-4f20-81e7-6462cfed7aa6 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:16:39.135180 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 965/1911] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 6 20:16:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:16:39.140686 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ff6c63-cfe2-4b5d-bd28-0b07852f8a7c None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:16:39.151488 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ff6c63-cfe2-4b5d-bd28-0b07852f8a7c None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:16:39.157010 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bc3c4f-2d4c-40fd-a6c1-76ae9175d5c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:16:39.158162 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bc3c4f-2d4c-40fd-a6c1-76ae9175d5c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:39.158443 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bc3c4f-2d4c-40fd-a6c1-76ae9175d5c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:39.158724 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bc3c4f-2d4c-40fd-a6c1-76ae9175d5c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:39.159416 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bc3c4f-2d4c-40fd-a6c1-76ae9175d5c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:16:39.161421 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bc3c4f-2d4c-40fd-a6c1-76ae9175d5c4 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:16:39.198828 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 947/1912] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:16:39 2022] POST /identity/v3/projects => generated 383 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:16:39.203468 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533b6212-62d4-4f20-81e7-6462cfed7aa6 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:16:39.215494 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533b6212-62d4-4f20-81e7-6462cfed7aa6 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:16:39.227584 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:16:39.293429 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c220bcb5-2176-4686-b6ee-9c6f70e3b2a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:16:39.294137 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c220bcb5-2176-4686-b6ee-9c6f70e3b2a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:39.294425 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c220bcb5-2176-4686-b6ee-9c6f70e3b2a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:39.294724 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c220bcb5-2176-4686-b6ee-9c6f70e3b2a5 admin admin] PATH_INFO: `/v3/users` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:39.295560 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c220bcb5-2176-4686-b6ee-9c6f70e3b2a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:16:39.297976 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c220bcb5-2176-4686-b6ee-9c6f70e3b2a5 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:16:39.326543 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 966/1913] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:16:39 2022] POST /identity/v3/users => generated 292 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:16:39.331735 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bc3c4f-2d4c-40fd-a6c1-76ae9175d5c4 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:16:39.344075 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bc3c4f-2d4c-40fd-a6c1-76ae9175d5c4 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:16:39.349761 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b65f48-e430-43bb-86b2-acce5e83c609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:16:39.350581 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b65f48-e430-43bb-86b2-acce5e83c609 admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:39.350995 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b65f48-e430-43bb-86b2-acce5e83c609 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:39.351546 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b65f48-e430-43bb-86b2-acce5e83c609 admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:39.352540 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b65f48-e430-43bb-86b2-acce5e83c609 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:16:39.356645 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b65f48-e430-43bb-86b2-acce5e83c609 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:16:39.371863 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 948/1914] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:16:39 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 20:16:39.377907 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c220bcb5-2176-4686-b6ee-9c6f70e3b2a5 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:16:39.395704 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c220bcb5-2176-4686-b6ee-9c6f70e3b2a5 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:16:39.401878 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4d332c-5ac8-45ee-9a17-c96f06ff56ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:16:39.402701 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4d332c-5ac8-45ee-9a17-c96f06ff56ca admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:39.403116 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4d332c-5ac8-45ee-9a17-c96f06ff56ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:39.403620 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4d332c-5ac8-45ee-9a17-c96f06ff56ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:39.405232 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4d332c-5ac8-45ee-9a17-c96f06ff56ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:16:39.406720 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4d332c-5ac8-45ee-9a17-c96f06ff56ca admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:16:39.422879 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 967/1915] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:16:39 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:16:39.429852 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b65f48-e430-43bb-86b2-acce5e83c609 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:16:39.442114 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b65f48-e430-43bb-86b2-acce5e83c609 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:16:39.448279 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f90411-f651-4ec4-ba43-be09bbd695a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:16:39.450830 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f90411-f651-4ec4-ba43-be09bbd695a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:39.451186 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f90411-f651-4ec4-ba43-be09bbd695a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:39.451535 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f90411-f651-4ec4-ba43-be09bbd695a4 admin admin] PATH_INFO: `/v3/projects/8f453e7b975445afaf2a9a0a82804f21/users/08dddbf517314eee8b20f76aaa01ac90/roles/4eea39e24b06431aa357053fe7907a3d` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:39.480645 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f90411-f651-4ec4-ba43-be09bbd695a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f453e7b975445afaf2a9a0a82804f21, user_id=08dddbf517314eee8b20f76aaa01ac90, role_id=4eea39e24b06431aa357053fe7907a3d)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:16:39.486265 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f90411-f651-4ec4-ba43-be09bbd695a4 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:16:39.511604 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 949/1916] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:16:39 2022] PUT /identity/v3/projects/8f453e7b975445afaf2a9a0a82804f21/users/08dddbf517314eee8b20f76aaa01ac90/roles/4eea39e24b06431aa357053fe7907a3d => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:16:39.535350 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dfa94a-ae7d-4b56-a0b0-bca5d2e765ff None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:39.535828 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dfa94a-ae7d-4b56-a0b0-bca5d2e765ff None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:39.536205 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dfa94a-ae7d-4b56-a0b0-bca5d2e765ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:39.600574 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-74dfa94a-ae7d-4b56-a0b0-bca5d2e765ff None None] MFA Rules not processed for user `08dddbf517314eee8b20f76aaa01ac90`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:16:39.605151 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-74dfa94a-ae7d-4b56-a0b0-bca5d2e765ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:16:39.746221 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 968/1917] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:16:39 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:16:39.747820 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f90411-f651-4ec4-ba43-be09bbd695a4 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:16:39.829945 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f90411-f651-4ec4-ba43-be09bbd695a4 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:16:39.836106 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf27394c-2bf4-4b37-9515-90dd7a6dcf41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:16:39.837002 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf27394c-2bf4-4b37-9515-90dd7a6dcf41 admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:39.837002 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf27394c-2bf4-4b37-9515-90dd7a6dcf41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:39.837344 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf27394c-2bf4-4b37-9515-90dd7a6dcf41 admin admin] PATH_INFO: `/v3/projects` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:39.837961 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf27394c-2bf4-4b37-9515-90dd7a6dcf41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:16:39.839905 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf27394c-2bf4-4b37-9515-90dd7a6dcf41 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:16:39.891139 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 950/1918] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:16:39 2022] POST /identity/v3/projects => generated 383 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 20:16:39.896072 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dfa94a-ae7d-4b56-a0b0-bca5d2e765ff None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:16:39.909802 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dfa94a-ae7d-4b56-a0b0-bca5d2e765ff None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:16:39.985487 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b3ee5f-3f85-4a8f-ba08-1dc12e1fb1df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:16:39.986643 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b3ee5f-3f85-4a8f-ba08-1dc12e1fb1df admin admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:39.986969 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b3ee5f-3f85-4a8f-ba08-1dc12e1fb1df admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:39.987314 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b3ee5f-3f85-4a8f-ba08-1dc12e1fb1df admin admin] PATH_INFO: `/v3/users` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:39.988046 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b3ee5f-3f85-4a8f-ba08-1dc12e1fb1df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:16:39.990205 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b3ee5f-3f85-4a8f-ba08-1dc12e1fb1df admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:16:40.025852 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 969/1919] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:16:39 2022] POST /identity/v3/users => generated 292 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:16:40.030822 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf27394c-2bf4-4b37-9515-90dd7a6dcf41 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:16:40.044164 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf27394c-2bf4-4b37-9515-90dd7a6dcf41 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:16:40.053316 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285e9dd6-2f2e-48b3-96d1-db34c1070538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:16:40.054663 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285e9dd6-2f2e-48b3-96d1-db34c1070538 admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:40.055444 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285e9dd6-2f2e-48b3-96d1-db34c1070538 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:40.055773 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285e9dd6-2f2e-48b3-96d1-db34c1070538 admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:40.056883 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285e9dd6-2f2e-48b3-96d1-db34c1070538 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:16:40.059294 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285e9dd6-2f2e-48b3-96d1-db34c1070538 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:16:40.074011 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 951/1920] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:16:40 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 20:16:40.078554 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b3ee5f-3f85-4a8f-ba08-1dc12e1fb1df admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:16:40.092548 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b3ee5f-3f85-4a8f-ba08-1dc12e1fb1df admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:16:40.101802 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fd1160-4722-45c1-8277-dfc408653614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:16:40.102455 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fd1160-4722-45c1-8277-dfc408653614 admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:40.102839 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fd1160-4722-45c1-8277-dfc408653614 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:40.103260 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fd1160-4722-45c1-8277-dfc408653614 admin admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:40.104194 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fd1160-4722-45c1-8277-dfc408653614 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:16:40.106305 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fd1160-4722-45c1-8277-dfc408653614 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:16:40.115152 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:16:40.124236 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 970/1921] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:16:40 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 20:16:40.129201 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285e9dd6-2f2e-48b3-96d1-db34c1070538 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:16:40.148364 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285e9dd6-2f2e-48b3-96d1-db34c1070538 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:16:40.156320 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddc2d54-254e-4d32-8839-0f0d62f5263f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:16:40.157194 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddc2d54-254e-4d32-8839-0f0d62f5263f admin admin] REQUEST_METHOD: `PUT` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:40.157810 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddc2d54-254e-4d32-8839-0f0d62f5263f admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:40.158109 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddc2d54-254e-4d32-8839-0f0d62f5263f admin admin] PATH_INFO: `/v3/projects/9ee600f98909456fb66ec7d197eb6d98/users/d1e8e46958c840d08f3e5487b935505e/roles/4eea39e24b06431aa357053fe7907a3d` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:40.182077 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddc2d54-254e-4d32-8839-0f0d62f5263f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ee600f98909456fb66ec7d197eb6d98, user_id=d1e8e46958c840d08f3e5487b935505e, role_id=4eea39e24b06431aa357053fe7907a3d)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:16:40.184837 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddc2d54-254e-4d32-8839-0f0d62f5263f admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:16:40.208975 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 952/1922] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:16:40 2022] PUT /identity/v3/projects/9ee600f98909456fb66ec7d197eb6d98/users/d1e8e46958c840d08f3e5487b935505e/roles/4eea39e24b06431aa357053fe7907a3d => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:16:40.223403 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cea3aa-9094-43b4-a173-04090e51966a None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:40.223976 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cea3aa-9094-43b4-a173-04090e51966a None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:40.224380 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cea3aa-9094-43b4-a173-04090e51966a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:40.261675 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-10cea3aa-9094-43b4-a173-04090e51966a None None] MFA Rules not processed for user `d1e8e46958c840d08f3e5487b935505e`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:16:40.262832 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-10cea3aa-9094-43b4-a173-04090e51966a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:16:40.353486 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 971/1923] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:16:40 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:16:40.359231 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddc2d54-254e-4d32-8839-0f0d62f5263f admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:16:40.421601 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:16:40.453322 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddc2d54-254e-4d32-8839-0f0d62f5263f admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:16:40.461733 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50948ce2-d0b8-487a-8f4a-3a677ef8bd06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:16:40.462789 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50948ce2-d0b8-487a-8f4a-3a677ef8bd06 admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:40.463282 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50948ce2-d0b8-487a-8f4a-3a677ef8bd06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:40.463766 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50948ce2-d0b8-487a-8f4a-3a677ef8bd06 admin admin] PATH_INFO: `/v3/projects` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:40.464932 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50948ce2-d0b8-487a-8f4a-3a677ef8bd06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:16:40.468776 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50948ce2-d0b8-487a-8f4a-3a677ef8bd06 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:16:40.505942 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 953/1924] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:16:40 2022] POST /identity/v3/projects => generated 385 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:16:40.512522 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cea3aa-9094-43b4-a173-04090e51966a None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:16:40.525501 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cea3aa-9094-43b4-a173-04090e51966a None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:16:40.601096 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bcf73f-4d38-4448-aa86-eca85c7e1a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:16:40.601597 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bcf73f-4d38-4448-aa86-eca85c7e1a8c admin admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:40.601921 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bcf73f-4d38-4448-aa86-eca85c7e1a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:40.602184 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bcf73f-4d38-4448-aa86-eca85c7e1a8c admin admin] PATH_INFO: `/v3/users` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:40.602869 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bcf73f-4d38-4448-aa86-eca85c7e1a8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:16:40.605560 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bcf73f-4d38-4448-aa86-eca85c7e1a8c admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:16:40.635064 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 972/1925] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:16:40 2022] POST /identity/v3/users => generated 299 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 20:16:40.640196 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50948ce2-d0b8-487a-8f4a-3a677ef8bd06 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:16:40.654749 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50948ce2-d0b8-487a-8f4a-3a677ef8bd06 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:16:40.676425 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e125ab5-8266-4d80-9955-f65deb4f8872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:16:40.677353 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e125ab5-8266-4d80-9955-f65deb4f8872 admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:40.677822 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e125ab5-8266-4d80-9955-f65deb4f8872 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:40.678248 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e125ab5-8266-4d80-9955-f65deb4f8872 admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:40.681308 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e125ab5-8266-4d80-9955-f65deb4f8872 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:16:40.684307 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e125ab5-8266-4d80-9955-f65deb4f8872 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:16:40.729804 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 954/1926] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:16:40 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:16:40.731188 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bcf73f-4d38-4448-aa86-eca85c7e1a8c admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:16:40.754866 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bcf73f-4d38-4448-aa86-eca85c7e1a8c admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:16:40.765417 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5989efa5-227b-4f87-be1c-24336b2d0516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:16:40.768046 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5989efa5-227b-4f87-be1c-24336b2d0516 admin admin] REQUEST_METHOD: `PUT` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:40.768654 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5989efa5-227b-4f87-be1c-24336b2d0516 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:40.769249 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5989efa5-227b-4f87-be1c-24336b2d0516 admin admin] PATH_INFO: `/v3/projects/13f06f8ba9d24869ac4fe2ea1c9f3379/users/be1005dff3ea41b197a6b9ee121b826e/roles/e0ab41d8ae22474688b5361bcc75e0f0` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:40.816049 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5989efa5-227b-4f87-be1c-24336b2d0516 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13f06f8ba9d24869ac4fe2ea1c9f3379, user_id=be1005dff3ea41b197a6b9ee121b826e, role_id=e0ab41d8ae22474688b5361bcc75e0f0)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:16:40.820626 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5989efa5-227b-4f87-be1c-24336b2d0516 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:16:40.839915 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 973/1927] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:16:40 2022] PUT /identity/v3/projects/13f06f8ba9d24869ac4fe2ea1c9f3379/users/be1005dff3ea41b197a6b9ee121b826e/roles/e0ab41d8ae22474688b5361bcc75e0f0 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 20:16:40.845708 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e125ab5-8266-4d80-9955-f65deb4f8872 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:16:40.865271 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e125ab5-8266-4d80-9955-f65deb4f8872 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:16:40.897899 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5989efa5-227b-4f87-be1c-24336b2d0516 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:16:40.900393 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-5989efa5-227b-4f87-be1c-24336b2d0516 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:16:40.989364 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1ba1b9-365b-4cf9-b909-6380d01bf2a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:16:40.990462 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1ba1b9-365b-4cf9-b909-6380d01bf2a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:40.991178 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1ba1b9-365b-4cf9-b909-6380d01bf2a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:40.991638 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1ba1b9-365b-4cf9-b909-6380d01bf2a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:40.992789 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1ba1b9-365b-4cf9-b909-6380d01bf2a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:16:40.995840 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1ba1b9-365b-4cf9-b909-6380d01bf2a0 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:16:41.010770 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 955/1928] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:16:40 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:16:41.014582 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:16:41.018277 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1ba1b9-365b-4cf9-b909-6380d01bf2a0 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:16:41.023788 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5989efa5-227b-4f87-be1c-24336b2d0516 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:16:41.031969 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1ba1b9-365b-4cf9-b909-6380d01bf2a0 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:16:41.032949 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc43d2e7-b022-4413-9170-1bdaecac14f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:16:41.033874 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc43d2e7-b022-4413-9170-1bdaecac14f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:41.034334 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc43d2e7-b022-4413-9170-1bdaecac14f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:41.034740 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc43d2e7-b022-4413-9170-1bdaecac14f5 admin admin] PATH_INFO: `/v3/users/a5f69414adb942c69a5e48b2cb94ceaf` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:41.036280 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc43d2e7-b022-4413-9170-1bdaecac14f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5f69414adb942c69a5e48b2cb94ceaf)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:16:41.038592 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc43d2e7-b022-4413-9170-1bdaecac14f5 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:16:41.040454 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d062bf5-d362-4079-8595-d59cab68718b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:16:41.041505 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d062bf5-d362-4079-8595-d59cab68718b admin admin] REQUEST_METHOD: `PUT` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:41.041981 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d062bf5-d362-4079-8595-d59cab68718b admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:41.042421 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d062bf5-d362-4079-8595-d59cab68718b admin admin] PATH_INFO: `/v3/projects/13f06f8ba9d24869ac4fe2ea1c9f3379/users/be1005dff3ea41b197a6b9ee121b826e/roles/e0ab41d8ae22474688b5361bcc75e0f0` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:41.045868 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d062bf5-d362-4079-8595-d59cab68718b admin admin] RBAC: Authorizing `identity:create_grant(project_id=13f06f8ba9d24869ac4fe2ea1c9f3379, user_id=be1005dff3ea41b197a6b9ee121b826e, role_id=e0ab41d8ae22474688b5361bcc75e0f0)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:16:41.051206 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d062bf5-d362-4079-8595-d59cab68718b admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:16:41.072540 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 956/1929] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:16:41 2022] PUT /identity/v3/projects/13f06f8ba9d24869ac4fe2ea1c9f3379/users/be1005dff3ea41b197a6b9ee121b826e/roles/e0ab41d8ae22474688b5361bcc75e0f0 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:16:41.088629 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed81ff3f-64b1-414d-85e1-33ebdb686700 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:41.089191 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed81ff3f-64b1-414d-85e1-33ebdb686700 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:41.089391 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed81ff3f-64b1-414d-85e1-33ebdb686700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:41.116728 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-fc43d2e7-b022-4413-9170-1bdaecac14f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5f69414adb942c69a5e48b2cb94ceaf'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:16:41.118185 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-fc43d2e7-b022-4413-9170-1bdaecac14f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5f69414adb942c69a5e48b2cb94ceaf'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:16:41.139423 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-ed81ff3f-64b1-414d-85e1-33ebdb686700 None None] MFA Rules not processed for user `be1005dff3ea41b197a6b9ee121b826e`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:16:41.140720 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-ed81ff3f-64b1-414d-85e1-33ebdb686700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:16:41.141965 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-fc43d2e7-b022-4413-9170-1bdaecac14f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5f69414adb942c69a5e48b2cb94ceaf'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:16:41.143224 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-fc43d2e7-b022-4413-9170-1bdaecac14f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5f69414adb942c69a5e48b2cb94ceaf'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:16:41.155579 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-fc43d2e7-b022-4413-9170-1bdaecac14f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5f69414adb942c69a5e48b2cb94ceaf'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:16:41.177839 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 974/1930] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:16:40 2022] DELETE /identity/v3/users/a5f69414adb942c69a5e48b2cb94ceaf => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 20:16:41.183370 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3d098e-1722-45a7-8910-262a8a04b0e2 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:41.183816 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3d098e-1722-45a7-8910-262a8a04b0e2 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:41.184239 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3d098e-1722-45a7-8910-262a8a04b0e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:41.216254 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-1c3d098e-1722-45a7-8910-262a8a04b0e2 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:16:41.218075 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-1c3d098e-1722-45a7-8910-262a8a04b0e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:16:41.253712 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 957/1931] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:16:41 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:16:41.273089 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed81ff3f-64b1-414d-85e1-33ebdb686700 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:16:41.275129 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-ed81ff3f-64b1-414d-85e1-33ebdb686700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:16:41.308823 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 975/1932] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:16:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:16:41.321774 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3d098e-1722-45a7-8910-262a8a04b0e2 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:16:41.323337 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-1c3d098e-1722-45a7-8910-262a8a04b0e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:16:41.332143 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:16:41.370267 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed81ff3f-64b1-414d-85e1-33ebdb686700 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:16:41.428788 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3d098e-1722-45a7-8910-262a8a04b0e2 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:16:41.436235 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdfc89e-cd7a-4e0d-ba8f-5e1c7eb10add service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:16:41.437272 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdfc89e-cd7a-4e0d-ba8f-5e1c7eb10add service neutron] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:41.437898 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdfc89e-cd7a-4e0d-ba8f-5e1c7eb10add service neutron] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:41.438389 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdfc89e-cd7a-4e0d-ba8f-5e1c7eb10add service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:41.442532 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bd784e-5620-4186-9381-de590294b0b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df1bca993bc142908cc7cf85eabd2b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:16:41.443646 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bd784e-5620-4186-9381-de590294b0b7 service cinder] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:41.444153 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bd784e-5620-4186-9381-de590294b0b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:41.444663 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bd784e-5620-4186-9381-de590294b0b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:41.447686 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-54bd784e-5620-4186-9381-de590294b0b7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:16:41.455066 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdfc89e-cd7a-4e0d-ba8f-5e1c7eb10add service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:16:41.456462 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdfc89e-cd7a-4e0d-ba8f-5e1c7eb10add service neutron] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:16:41.468118 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 976/1933] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:16:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 20:16:41.542660 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bd784e-5620-4186-9381-de590294b0b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:16:41.544567 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bd784e-5620-4186-9381-de590294b0b7 service cinder] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:16:41.562033 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 958/1934] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:16:41 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 20:16:42.636747 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdfc89e-cd7a-4e0d-ba8f-5e1c7eb10add service neutron] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:16:42.639075 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-ccdfc89e-cd7a-4e0d-ba8f-5e1c7eb10add service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:16:42.665082 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdfc89e-cd7a-4e0d-ba8f-5e1c7eb10add service neutron] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:16:42.673760 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e93d30a-bf91-4352-8af9-6d5e95343c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:16:42.674859 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e93d30a-bf91-4352-8af9-6d5e95343c82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:42.675563 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e93d30a-bf91-4352-8af9-6d5e95343c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:42.675772 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e93d30a-bf91-4352-8af9-6d5e95343c82 admin admin] PATH_INFO: `/v3/projects/1eb1b878dcf248639e486c467de8378d` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:42.678163 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e93d30a-bf91-4352-8af9-6d5e95343c82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1eb1b878dcf248639e486c467de8378d)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:16:42.682016 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e93d30a-bf91-4352-8af9-6d5e95343c82 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:16:42.708368 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:16:42.775876 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-7e93d30a-bf91-4352-8af9-6d5e95343c82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1eb1b878dcf248639e486c467de8378d', 'request_id': 'req-7e93d30a-bf91-4352-8af9-6d5e95343c82'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:16:42.788707 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-7e93d30a-bf91-4352-8af9-6d5e95343c82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1eb1b878dcf248639e486c467de8378d', 'request_id': 'req-7e93d30a-bf91-4352-8af9-6d5e95343c82'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:16:42.799333 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-7e93d30a-bf91-4352-8af9-6d5e95343c82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1eb1b878dcf248639e486c467de8378d', 'request_id': 'req-7e93d30a-bf91-4352-8af9-6d5e95343c82'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:16:42.819848 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-7e93d30a-bf91-4352-8af9-6d5e95343c82 admin admin] The token cache is being invalidate because project 1eb1b878dcf248639e486c467de8378d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81772) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:16:42.820608 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-7e93d30a-bf91-4352-8af9-6d5e95343c82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:16:42.821672 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-7e93d30a-bf91-4352-8af9-6d5e95343c82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:16:42.823881 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 977/1935] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:16:42 2022] DELETE /identity/v3/projects/1eb1b878dcf248639e486c467de8378d => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:16:42.831579 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d054fc-700a-4fc4-9782-7ec0657a7142 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:42.832129 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d054fc-700a-4fc4-9782-7ec0657a7142 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:42.832524 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d054fc-700a-4fc4-9782-7ec0657a7142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:42.869285 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-48d054fc-700a-4fc4-9782-7ec0657a7142 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:16:42.871518 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-48d054fc-700a-4fc4-9782-7ec0657a7142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:16:42.984887 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 959/1936] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:16:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:16:43.000025 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d13687-a961-4746-a34d-d12b6aa367a6 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:43.000025 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d13687-a961-4746-a34d-d12b6aa367a6 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:43.000025 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d13687-a961-4746-a34d-d12b6aa367a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:43.034057 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-39d13687-a961-4746-a34d-d12b6aa367a6 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:16:43.036462 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-39d13687-a961-4746-a34d-d12b6aa367a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:16:43.060372 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 978/1937] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:16:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:16:43.066625 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d054fc-700a-4fc4-9782-7ec0657a7142 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:16:43.081148 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d054fc-700a-4fc4-9782-7ec0657a7142 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:16:43.096377 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e70a870-b76a-4f07-8ad7-6b133cf39350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:16:43.097602 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e70a870-b76a-4f07-8ad7-6b133cf39350 admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:43.098101 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e70a870-b76a-4f07-8ad7-6b133cf39350 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:43.098576 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e70a870-b76a-4f07-8ad7-6b133cf39350 admin admin] PATH_INFO: `/v3/domains` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:43.099763 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e70a870-b76a-4f07-8ad7-6b133cf39350 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:16:43.101379 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e70a870-b76a-4f07-8ad7-6b133cf39350 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:16:43.108068 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e70a870-b76a-4f07-8ad7-6b133cf39350 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:16:43.154968 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 960/1938] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 6 20:16:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:16:43.156710 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d13687-a961-4746-a34d-d12b6aa367a6 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:16:43.173864 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d13687-a961-4746-a34d-d12b6aa367a6 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:16:43.183179 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3abfcf-076e-4312-9c57-2a3796419af5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:16:43.184211 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3abfcf-076e-4312-9c57-2a3796419af5 admin admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:43.184727 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3abfcf-076e-4312-9c57-2a3796419af5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:43.186056 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3abfcf-076e-4312-9c57-2a3796419af5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:43.187263 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3abfcf-076e-4312-9c57-2a3796419af5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:16:43.190023 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3abfcf-076e-4312-9c57-2a3796419af5 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:16:43.244693 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 979/1939] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:16:43 2022] POST /identity/v3/projects => generated 365 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:16:43.249429 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e70a870-b76a-4f07-8ad7-6b133cf39350 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:16:43.274136 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e70a870-b76a-4f07-8ad7-6b133cf39350 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:16:43.316046 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:16:43.437429 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3103c5b1-9575-49dc-b325-09c7d6410551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:16:43.438112 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3103c5b1-9575-49dc-b325-09c7d6410551 admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:43.438399 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3103c5b1-9575-49dc-b325-09c7d6410551 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:43.438679 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3103c5b1-9575-49dc-b325-09c7d6410551 admin admin] PATH_INFO: `/v3/users` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:43.442455 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3103c5b1-9575-49dc-b325-09c7d6410551 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:16:43.446153 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3103c5b1-9575-49dc-b325-09c7d6410551 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:16:43.498531 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 961/1940] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:16:43 2022] POST /identity/v3/users => generated 283 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 20:16:43.504392 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3abfcf-076e-4312-9c57-2a3796419af5 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:16:43.523984 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3abfcf-076e-4312-9c57-2a3796419af5 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:16:43.533583 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081f1f8a-4c0f-40da-a190-ded421495c7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:16:43.534831 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081f1f8a-4c0f-40da-a190-ded421495c7a admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:43.535377 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081f1f8a-4c0f-40da-a190-ded421495c7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:43.535941 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081f1f8a-4c0f-40da-a190-ded421495c7a admin admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:43.537431 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081f1f8a-4c0f-40da-a190-ded421495c7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:16:43.540780 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081f1f8a-4c0f-40da-a190-ded421495c7a admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:16:43.561138 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 980/1941] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:16:43 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:16:43.565361 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3103c5b1-9575-49dc-b325-09c7d6410551 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:16:43.565560 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081f1f8a-4c0f-40da-a190-ded421495c7a admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:16:43.568285 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-3103c5b1-9575-49dc-b325-09c7d6410551 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:16:43.577437 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081f1f8a-4c0f-40da-a190-ded421495c7a admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:16:43.584231 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251bd991-a518-46f7-892b-e406a9bc7bcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:16:43.584986 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251bd991-a518-46f7-892b-e406a9bc7bcd admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:43.585551 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251bd991-a518-46f7-892b-e406a9bc7bcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:43.585890 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251bd991-a518-46f7-892b-e406a9bc7bcd admin admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:43.587483 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251bd991-a518-46f7-892b-e406a9bc7bcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:16:43.588605 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251bd991-a518-46f7-892b-e406a9bc7bcd admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:16:43.602271 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 981/1942] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:16:43 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:16:43.606766 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251bd991-a518-46f7-892b-e406a9bc7bcd admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:16:43.622585 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251bd991-a518-46f7-892b-e406a9bc7bcd admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:16:43.630298 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a581455-8745-4ed0-a3a2-6312e2ed91a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:16:43.631144 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a581455-8745-4ed0-a3a2-6312e2ed91a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:43.631666 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a581455-8745-4ed0-a3a2-6312e2ed91a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:43.632237 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a581455-8745-4ed0-a3a2-6312e2ed91a5 admin admin] PATH_INFO: `/v3/projects/3883c8302985440897fbc397161f297d/users/9697e8514c1c4676ad67bcab474edd74/roles/4eea39e24b06431aa357053fe7907a3d` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:43.671700 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a581455-8745-4ed0-a3a2-6312e2ed91a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3883c8302985440897fbc397161f297d, user_id=9697e8514c1c4676ad67bcab474edd74, role_id=4eea39e24b06431aa357053fe7907a3d)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:16:43.676843 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a581455-8745-4ed0-a3a2-6312e2ed91a5 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:16:43.696846 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3103c5b1-9575-49dc-b325-09c7d6410551 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:16:43.700736 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 982/1943] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:16:43 2022] PUT /identity/v3/projects/3883c8302985440897fbc397161f297d/users/9697e8514c1c4676ad67bcab474edd74/roles/4eea39e24b06431aa357053fe7907a3d => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:16:43.716614 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bd2206-731f-45f7-933e-13b561054e11 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:43.717085 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bd2206-731f-45f7-933e-13b561054e11 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:43.717497 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bd2206-731f-45f7-933e-13b561054e11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:43.753431 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-23bd2206-731f-45f7-933e-13b561054e11 None None] MFA Rules not processed for user `9697e8514c1c4676ad67bcab474edd74`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:16:43.754850 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-23bd2206-731f-45f7-933e-13b561054e11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:16:43.776965 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716775b7-45b6-4431-b0da-e4de782db21a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df1bca993bc142908cc7cf85eabd2b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:16:43.778235 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716775b7-45b6-4431-b0da-e4de782db21a service cinder] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:43.778731 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716775b7-45b6-4431-b0da-e4de782db21a service cinder] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:43.779236 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716775b7-45b6-4431-b0da-e4de782db21a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:43.782858 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-716775b7-45b6-4431-b0da-e4de782db21a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:16:43.894616 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 983/1944] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:16:43 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:16:43.913587 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716775b7-45b6-4431-b0da-e4de782db21a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:16:43.915345 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bd2206-731f-45f7-933e-13b561054e11 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:16:43.915611 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716775b7-45b6-4431-b0da-e4de782db21a service cinder] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:16:43.918346 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-23bd2206-731f-45f7-933e-13b561054e11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:16:43.932879 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 962/1945] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:16:43 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:16:43.943392 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bd2206-731f-45f7-933e-13b561054e11 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:16:43.952263 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b016b1-b32c-4bed-b0df-d0d7f90ae6b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df1bca993bc142908cc7cf85eabd2b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:16:43.953383 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b016b1-b32c-4bed-b0df-d0d7f90ae6b5 service cinder] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:43.954005 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b016b1-b32c-4bed-b0df-d0d7f90ae6b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:43.955037 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b016b1-b32c-4bed-b0df-d0d7f90ae6b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:43.970229 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b016b1-b32c-4bed-b0df-d0d7f90ae6b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:16:43.972637 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b016b1-b32c-4bed-b0df-d0d7f90ae6b5 service cinder] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:16:43.989598 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 984/1946] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:16:43 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 20:16:44.745784 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716775b7-45b6-4431-b0da-e4de782db21a service cinder] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:16:44.761041 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716775b7-45b6-4431-b0da-e4de782db21a service cinder] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:16:44.769506 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04df323d-8d37-433b-86a1-19c8b53aada0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df1bca993bc142908cc7cf85eabd2b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:16:44.770786 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04df323d-8d37-433b-86a1-19c8b53aada0 service cinder] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:44.772321 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04df323d-8d37-433b-86a1-19c8b53aada0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:44.772321 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04df323d-8d37-433b-86a1-19c8b53aada0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:44.775674 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-04df323d-8d37-433b-86a1-19c8b53aada0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:16:44.827284 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:16:44.910711 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04df323d-8d37-433b-86a1-19c8b53aada0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:16:44.913888 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04df323d-8d37-433b-86a1-19c8b53aada0 service cinder] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:16:44.933290 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 963/1947] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:16:44 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:16:49.367651 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b016b1-b32c-4bed-b0df-d0d7f90ae6b5 service cinder] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:16:49.370091 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-f7b016b1-b32c-4bed-b0df-d0d7f90ae6b5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:16:49.380427 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:16:49.489465 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b016b1-b32c-4bed-b0df-d0d7f90ae6b5 service cinder] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:16:49.494609 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5acef8-316c-4862-970d-05141c3a75b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:16:49.495510 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5acef8-316c-4862-970d-05141c3a75b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:49.496001 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5acef8-316c-4862-970d-05141c3a75b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:49.496415 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5acef8-316c-4862-970d-05141c3a75b3 admin admin] PATH_INFO: `/v3/users/08dddbf517314eee8b20f76aaa01ac90` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:49.497791 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5acef8-316c-4862-970d-05141c3a75b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=08dddbf517314eee8b20f76aaa01ac90)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:16:49.499820 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5acef8-316c-4862-970d-05141c3a75b3 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:16:49.563284 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-2c5acef8-316c-4862-970d-05141c3a75b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08dddbf517314eee8b20f76aaa01ac90'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:16:49.564810 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-2c5acef8-316c-4862-970d-05141c3a75b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08dddbf517314eee8b20f76aaa01ac90'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:16:49.582140 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-2c5acef8-316c-4862-970d-05141c3a75b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08dddbf517314eee8b20f76aaa01ac90'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:16:49.582984 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-2c5acef8-316c-4862-970d-05141c3a75b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08dddbf517314eee8b20f76aaa01ac90'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:16:49.593590 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-2c5acef8-316c-4862-970d-05141c3a75b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08dddbf517314eee8b20f76aaa01ac90'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:16:49.609644 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 985/1948] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:16:49 2022] DELETE /identity/v3/users/08dddbf517314eee8b20f76aaa01ac90 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:16:49.615528 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca9d90f-7ad8-4c32-9866-1049396081a4 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:49.616379 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca9d90f-7ad8-4c32-9866-1049396081a4 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:49.616792 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca9d90f-7ad8-4c32-9866-1049396081a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:49.643346 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-fca9d90f-7ad8-4c32-9866-1049396081a4 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:16:49.645401 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-fca9d90f-7ad8-4c32-9866-1049396081a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:16:49.757259 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 964/1949] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:16:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:16:49.768307 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5acef8-316c-4862-970d-05141c3a75b3 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:16:49.770041 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-2c5acef8-316c-4862-970d-05141c3a75b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:16:49.847040 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5acef8-316c-4862-970d-05141c3a75b3 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:16:49.851153 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddea077d-8e01-4c58-ae28-f4d1145f9d2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:16:49.851818 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddea077d-8e01-4c58-ae28-f4d1145f9d2f service neutron] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:49.852202 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddea077d-8e01-4c58-ae28-f4d1145f9d2f service neutron] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:49.852374 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddea077d-8e01-4c58-ae28-f4d1145f9d2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:49.864400 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddea077d-8e01-4c58-ae28-f4d1145f9d2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:16:49.866431 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddea077d-8e01-4c58-ae28-f4d1145f9d2f service neutron] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:16:49.883547 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 986/1950] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:16:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 20:16:50.821467 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca9d90f-7ad8-4c32-9866-1049396081a4 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:16:50.824143 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-fca9d90f-7ad8-4c32-9866-1049396081a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:16:50.841462 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca9d90f-7ad8-4c32-9866-1049396081a4 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:16:50.850054 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fc991c-168e-4928-8986-3468bd44110a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:16:50.851120 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fc991c-168e-4928-8986-3468bd44110a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:50.851839 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fc991c-168e-4928-8986-3468bd44110a admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:50.852380 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fc991c-168e-4928-8986-3468bd44110a admin admin] PATH_INFO: `/v3/projects/8f453e7b975445afaf2a9a0a82804f21` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:50.854151 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fc991c-168e-4928-8986-3468bd44110a admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f453e7b975445afaf2a9a0a82804f21)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:16:50.857111 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fc991c-168e-4928-8986-3468bd44110a admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:16:50.962577 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-59fc991c-168e-4928-8986-3468bd44110a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f453e7b975445afaf2a9a0a82804f21', 'request_id': 'req-59fc991c-168e-4928-8986-3468bd44110a'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:16:50.974674 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-59fc991c-168e-4928-8986-3468bd44110a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f453e7b975445afaf2a9a0a82804f21', 'request_id': 'req-59fc991c-168e-4928-8986-3468bd44110a'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:16:50.988901 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-59fc991c-168e-4928-8986-3468bd44110a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f453e7b975445afaf2a9a0a82804f21', 'request_id': 'req-59fc991c-168e-4928-8986-3468bd44110a'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:16:51.013247 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-59fc991c-168e-4928-8986-3468bd44110a admin admin] The token cache is being invalidate because project 8f453e7b975445afaf2a9a0a82804f21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81771) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:16:51.013570 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-59fc991c-168e-4928-8986-3468bd44110a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:16:51.014888 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-59fc991c-168e-4928-8986-3468bd44110a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:16:51.017516 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 965/1951] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:16:50 2022] DELETE /identity/v3/projects/8f453e7b975445afaf2a9a0a82804f21 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:16:51.021988 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddea077d-8e01-4c58-ae28-f4d1145f9d2f service neutron] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:16:51.038780 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddea077d-8e01-4c58-ae28-f4d1145f9d2f service neutron] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:16:51.080618 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:16:51.173319 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e135a6c4-5c8a-4185-9744-34c94acaffb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:16:51.175168 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e135a6c4-5c8a-4185-9744-34c94acaffb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:51.175794 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e135a6c4-5c8a-4185-9744-34c94acaffb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:51.176381 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e135a6c4-5c8a-4185-9744-34c94acaffb9 admin admin] PATH_INFO: `/v3/users/d1e8e46958c840d08f3e5487b935505e` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:51.178901 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e135a6c4-5c8a-4185-9744-34c94acaffb9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1e8e46958c840d08f3e5487b935505e)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:16:51.183411 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e135a6c4-5c8a-4185-9744-34c94acaffb9 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:16:51.280504 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-e135a6c4-5c8a-4185-9744-34c94acaffb9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1e8e46958c840d08f3e5487b935505e'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:16:51.281326 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-e135a6c4-5c8a-4185-9744-34c94acaffb9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1e8e46958c840d08f3e5487b935505e'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:16:51.317956 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-e135a6c4-5c8a-4185-9744-34c94acaffb9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1e8e46958c840d08f3e5487b935505e'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:16:51.318712 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-e135a6c4-5c8a-4185-9744-34c94acaffb9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1e8e46958c840d08f3e5487b935505e'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:16:51.330880 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-e135a6c4-5c8a-4185-9744-34c94acaffb9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1e8e46958c840d08f3e5487b935505e'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:16:51.353205 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 987/1952] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:16:51 2022] DELETE /identity/v3/users/d1e8e46958c840d08f3e5487b935505e => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:16:52.102823 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fc991c-168e-4928-8986-3468bd44110a admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:16:52.105689 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-59fc991c-168e-4928-8986-3468bd44110a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:16:52.122554 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fc991c-168e-4928-8986-3468bd44110a admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:16:52.192125 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:16:52.224627 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab8d45f-58d1-4bc4-863a-5dfb36b75c00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:16:52.225537 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab8d45f-58d1-4bc4-863a-5dfb36b75c00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:52.226028 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab8d45f-58d1-4bc4-863a-5dfb36b75c00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:52.226450 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab8d45f-58d1-4bc4-863a-5dfb36b75c00 admin admin] PATH_INFO: `/v3/projects/9ee600f98909456fb66ec7d197eb6d98` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:52.227974 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab8d45f-58d1-4bc4-863a-5dfb36b75c00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ee600f98909456fb66ec7d197eb6d98)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:16:52.230006 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab8d45f-58d1-4bc4-863a-5dfb36b75c00 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:16:52.326774 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-3ab8d45f-58d1-4bc4-863a-5dfb36b75c00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ee600f98909456fb66ec7d197eb6d98', 'request_id': 'req-3ab8d45f-58d1-4bc4-863a-5dfb36b75c00'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:16:52.335986 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-3ab8d45f-58d1-4bc4-863a-5dfb36b75c00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ee600f98909456fb66ec7d197eb6d98', 'request_id': 'req-3ab8d45f-58d1-4bc4-863a-5dfb36b75c00'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:16:52.349970 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-3ab8d45f-58d1-4bc4-863a-5dfb36b75c00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ee600f98909456fb66ec7d197eb6d98', 'request_id': 'req-3ab8d45f-58d1-4bc4-863a-5dfb36b75c00'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:16:52.368844 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-3ab8d45f-58d1-4bc4-863a-5dfb36b75c00 admin admin] The token cache is being invalidate because project 9ee600f98909456fb66ec7d197eb6d98 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81771) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:16:52.369298 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-3ab8d45f-58d1-4bc4-863a-5dfb36b75c00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:16:52.370136 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-3ab8d45f-58d1-4bc4-863a-5dfb36b75c00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:16:52.372143 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 966/1953] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:16:52 2022] DELETE /identity/v3/projects/9ee600f98909456fb66ec7d197eb6d98 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:16:52.376695 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e135a6c4-5c8a-4185-9744-34c94acaffb9 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:16:52.379028 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-e135a6c4-5c8a-4185-9744-34c94acaffb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:16:52.394197 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e135a6c4-5c8a-4185-9744-34c94acaffb9 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:16:52.486498 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77219d9a-e6ba-4d00-b375-b754258645de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:16:52.487259 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77219d9a-e6ba-4d00-b375-b754258645de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:52.487547 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77219d9a-e6ba-4d00-b375-b754258645de admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:52.487866 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77219d9a-e6ba-4d00-b375-b754258645de admin admin] PATH_INFO: `/v3/users/be1005dff3ea41b197a6b9ee121b826e` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:52.489231 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77219d9a-e6ba-4d00-b375-b754258645de admin admin] RBAC: Authorizing `identity:delete_user(user_id=be1005dff3ea41b197a6b9ee121b826e)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:16:52.491531 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77219d9a-e6ba-4d00-b375-b754258645de admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:16:52.554087 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:16:52.557135 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-77219d9a-e6ba-4d00-b375-b754258645de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be1005dff3ea41b197a6b9ee121b826e'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:16:52.558114 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-77219d9a-e6ba-4d00-b375-b754258645de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be1005dff3ea41b197a6b9ee121b826e'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:16:52.582789 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-77219d9a-e6ba-4d00-b375-b754258645de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be1005dff3ea41b197a6b9ee121b826e'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:16:52.584234 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-77219d9a-e6ba-4d00-b375-b754258645de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be1005dff3ea41b197a6b9ee121b826e'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:16:52.596259 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-77219d9a-e6ba-4d00-b375-b754258645de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be1005dff3ea41b197a6b9ee121b826e'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:16:52.612768 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 988/1954] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:16:52 2022] DELETE /identity/v3/users/be1005dff3ea41b197a6b9ee121b826e => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:16:53.021867 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab8d45f-58d1-4bc4-863a-5dfb36b75c00 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:16:53.026484 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-3ab8d45f-58d1-4bc4-863a-5dfb36b75c00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:16:53.044522 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab8d45f-58d1-4bc4-863a-5dfb36b75c00 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:16:53.166990 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2867b9b6-a9b4-465f-b53d-60d810a26f73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:16:53.168156 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2867b9b6-a9b4-465f-b53d-60d810a26f73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:53.168905 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2867b9b6-a9b4-465f-b53d-60d810a26f73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:53.169636 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2867b9b6-a9b4-465f-b53d-60d810a26f73 admin admin] PATH_INFO: `/v3/projects/13f06f8ba9d24869ac4fe2ea1c9f3379` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:53.172321 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2867b9b6-a9b4-465f-b53d-60d810a26f73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=13f06f8ba9d24869ac4fe2ea1c9f3379)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:16:53.177130 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2867b9b6-a9b4-465f-b53d-60d810a26f73 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:16:53.265248 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-2867b9b6-a9b4-465f-b53d-60d810a26f73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13f06f8ba9d24869ac4fe2ea1c9f3379', 'request_id': 'req-2867b9b6-a9b4-465f-b53d-60d810a26f73'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:16:53.277917 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-2867b9b6-a9b4-465f-b53d-60d810a26f73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13f06f8ba9d24869ac4fe2ea1c9f3379', 'request_id': 'req-2867b9b6-a9b4-465f-b53d-60d810a26f73'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:16:53.294005 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-2867b9b6-a9b4-465f-b53d-60d810a26f73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13f06f8ba9d24869ac4fe2ea1c9f3379', 'request_id': 'req-2867b9b6-a9b4-465f-b53d-60d810a26f73'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:16:53.318619 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-2867b9b6-a9b4-465f-b53d-60d810a26f73 admin admin] The token cache is being invalidate because project 13f06f8ba9d24869ac4fe2ea1c9f3379 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81771) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:16:53.319096 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-2867b9b6-a9b4-465f-b53d-60d810a26f73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:16:53.320843 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-2867b9b6-a9b4-465f-b53d-60d810a26f73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:16:53.323447 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 967/1955] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:16:53 2022] DELETE /identity/v3/projects/13f06f8ba9d24869ac4fe2ea1c9f3379 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:16:53.332807 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61e1d52-455a-4824-adfa-e8a21112307f None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:53.334231 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61e1d52-455a-4824-adfa-e8a21112307f None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:53.334734 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61e1d52-455a-4824-adfa-e8a21112307f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:53.359927 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-f61e1d52-455a-4824-adfa-e8a21112307f None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:16:53.361756 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-f61e1d52-455a-4824-adfa-e8a21112307f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:16:53.445056 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 989/1956] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:16:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:16:53.459206 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6c1c32-c61c-49c4-b685-76e6600bffa6 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:53.459609 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6c1c32-c61c-49c4-b685-76e6600bffa6 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:53.460040 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6c1c32-c61c-49c4-b685-76e6600bffa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:53.480700 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:16:53.490460 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-ea6c1c32-c61c-49c4-b685-76e6600bffa6 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:16:53.494675 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-ea6c1c32-c61c-49c4-b685-76e6600bffa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:16:53.506675 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 968/1957] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:16:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:16:53.511344 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61e1d52-455a-4824-adfa-e8a21112307f None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:16:53.524216 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61e1d52-455a-4824-adfa-e8a21112307f None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:16:53.531311 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18c0c3c-767f-43b6-967e-ea61a22b9363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:16:53.532000 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18c0c3c-767f-43b6-967e-ea61a22b9363 admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:53.532381 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18c0c3c-767f-43b6-967e-ea61a22b9363 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:53.532658 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18c0c3c-767f-43b6-967e-ea61a22b9363 admin admin] PATH_INFO: `/v3/domains` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:53.533363 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18c0c3c-767f-43b6-967e-ea61a22b9363 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:16:53.533760 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18c0c3c-767f-43b6-967e-ea61a22b9363 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:16:53.535913 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18c0c3c-767f-43b6-967e-ea61a22b9363 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:16:53.551725 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 990/1958] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 6 20:16:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:16:53.562263 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6c1c32-c61c-49c4-b685-76e6600bffa6 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:16:53.574079 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6c1c32-c61c-49c4-b685-76e6600bffa6 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:16:53.580887 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf92574a-dc65-4e4d-ba4d-d2e58e1e4141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:16:53.581649 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf92574a-dc65-4e4d-ba4d-d2e58e1e4141 admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:53.581998 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf92574a-dc65-4e4d-ba4d-d2e58e1e4141 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:53.582344 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf92574a-dc65-4e4d-ba4d-d2e58e1e4141 admin admin] PATH_INFO: `/v3/projects` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:53.583081 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf92574a-dc65-4e4d-ba4d-d2e58e1e4141 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:16:53.585088 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf92574a-dc65-4e4d-ba4d-d2e58e1e4141 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:16:53.634833 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 969/1959] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:16:53 2022] POST /identity/v3/projects => generated 381 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:16:53.637087 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18c0c3c-767f-43b6-967e-ea61a22b9363 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:16:53.655393 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18c0c3c-767f-43b6-967e-ea61a22b9363 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:16:53.797578 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ee7fe2-e68c-4c3e-82a4-2a65e659a0e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:16:53.799372 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ee7fe2-e68c-4c3e-82a4-2a65e659a0e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:53.801037 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ee7fe2-e68c-4c3e-82a4-2a65e659a0e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:53.801890 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ee7fe2-e68c-4c3e-82a4-2a65e659a0e6 admin admin] PATH_INFO: `/v3/users` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:53.803340 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ee7fe2-e68c-4c3e-82a4-2a65e659a0e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:16:53.809178 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ee7fe2-e68c-4c3e-82a4-2a65e659a0e6 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:16:53.856593 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 991/1960] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:16:53 2022] POST /identity/v3/users => generated 291 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:16:53.864096 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf92574a-dc65-4e4d-ba4d-d2e58e1e4141 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:16:53.886958 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf92574a-dc65-4e4d-ba4d-d2e58e1e4141 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:16:53.895714 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2bccd6-dafe-407d-85f6-2746d4b86bcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:16:53.897103 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2bccd6-dafe-407d-85f6-2746d4b86bcf admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:53.897987 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2bccd6-dafe-407d-85f6-2746d4b86bcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:53.898530 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2bccd6-dafe-407d-85f6-2746d4b86bcf admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:53.899848 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2bccd6-dafe-407d-85f6-2746d4b86bcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:16:53.903365 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2bccd6-dafe-407d-85f6-2746d4b86bcf admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:16:53.928735 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 970/1961] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:16:53 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:16:53.930176 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ee7fe2-e68c-4c3e-82a4-2a65e659a0e6 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:16:53.942465 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ee7fe2-e68c-4c3e-82a4-2a65e659a0e6 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:16:53.948563 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647c28f8-7f91-4a41-b66f-3f61da1d0495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:16:53.949276 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647c28f8-7f91-4a41-b66f-3f61da1d0495 admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:53.949704 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647c28f8-7f91-4a41-b66f-3f61da1d0495 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:53.949926 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647c28f8-7f91-4a41-b66f-3f61da1d0495 admin admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:53.950826 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647c28f8-7f91-4a41-b66f-3f61da1d0495 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:16:53.952891 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647c28f8-7f91-4a41-b66f-3f61da1d0495 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:16:53.968755 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 992/1962] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:16:53 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:16:53.974312 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2bccd6-dafe-407d-85f6-2746d4b86bcf admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:16:53.989459 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2bccd6-dafe-407d-85f6-2746d4b86bcf admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:16:53.995169 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fcc312-3ac0-42ee-8fc5-7cfe22924eed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:16:53.996304 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fcc312-3ac0-42ee-8fc5-7cfe22924eed admin admin] REQUEST_METHOD: `PUT` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:53.996763 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fcc312-3ac0-42ee-8fc5-7cfe22924eed admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:53.997272 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fcc312-3ac0-42ee-8fc5-7cfe22924eed admin admin] PATH_INFO: `/v3/projects/b5de971d8037434fbaf3c5aa93729028/users/87f8cc79586f4cb79c962ede8342f580/roles/4eea39e24b06431aa357053fe7907a3d` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:54.017122 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fcc312-3ac0-42ee-8fc5-7cfe22924eed admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5de971d8037434fbaf3c5aa93729028, user_id=87f8cc79586f4cb79c962ede8342f580, role_id=4eea39e24b06431aa357053fe7907a3d)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:16:54.020029 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fcc312-3ac0-42ee-8fc5-7cfe22924eed admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:16:54.040165 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 971/1963] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:16:53 2022] PUT /identity/v3/projects/b5de971d8037434fbaf3c5aa93729028/users/87f8cc79586f4cb79c962ede8342f580/roles/4eea39e24b06431aa357053fe7907a3d => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:16:54.045640 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee3c5c2-85f5-4eed-a796-c6af455005d0 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:54.046409 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee3c5c2-85f5-4eed-a796-c6af455005d0 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:54.046869 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee3c5c2-85f5-4eed-a796-c6af455005d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:54.072490 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-6ee3c5c2-85f5-4eed-a796-c6af455005d0 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:16:54.073806 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-6ee3c5c2-85f5-4eed-a796-c6af455005d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:16:54.111528 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:16:54.166610 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 993/1964] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:16:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:16:54.177991 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fcc312-3ac0-42ee-8fc5-7cfe22924eed admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:16:54.180052 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-b4fcc312-3ac0-42ee-8fc5-7cfe22924eed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:16:54.253526 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fcc312-3ac0-42ee-8fc5-7cfe22924eed admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:16:54.258732 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff75bc6-7fe4-4714-921c-6a071b081bc1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:16:54.259388 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff75bc6-7fe4-4714-921c-6a071b081bc1 service neutron] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:54.259674 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff75bc6-7fe4-4714-921c-6a071b081bc1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:54.259953 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff75bc6-7fe4-4714-921c-6a071b081bc1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:54.271299 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff75bc6-7fe4-4714-921c-6a071b081bc1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:16:54.273052 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff75bc6-7fe4-4714-921c-6a071b081bc1 service neutron] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:16:54.285182 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 972/1965] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:16:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:16:55.810518 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cf099d-e25b-4462-9074-0e4ac3f229ef None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:55.811232 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cf099d-e25b-4462-9074-0e4ac3f229ef None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:55.811232 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cf099d-e25b-4462-9074-0e4ac3f229ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:55.849273 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-34cf099d-e25b-4462-9074-0e4ac3f229ef None None] MFA Rules not processed for user `87f8cc79586f4cb79c962ede8342f580`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:16:55.850812 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-34cf099d-e25b-4462-9074-0e4ac3f229ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:16:55.977959 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 994/1966] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:16:55 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:16:55.995881 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff75bc6-7fe4-4714-921c-6a071b081bc1 service neutron] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:16:56.002591 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-9ff75bc6-7fe4-4714-921c-6a071b081bc1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:16:56.221366 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff75bc6-7fe4-4714-921c-6a071b081bc1 service neutron] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:16:56.231958 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98211653-a962-49ca-a9f4-d3ae3cae0def service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df1bca993bc142908cc7cf85eabd2b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:16:56.234356 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98211653-a962-49ca-a9f4-d3ae3cae0def service cinder] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:56.234877 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98211653-a962-49ca-a9f4-d3ae3cae0def service cinder] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:56.235473 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98211653-a962-49ca-a9f4-d3ae3cae0def service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:56.258592 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98211653-a962-49ca-a9f4-d3ae3cae0def service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:16:56.272007 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98211653-a962-49ca-a9f4-d3ae3cae0def service cinder] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:16:56.312375 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 973/1967] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:16:55 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 20:16:58.355921 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf0cc16-5dc7-42b0-985f-5def6d6035e4 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:58.356622 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf0cc16-5dc7-42b0-985f-5def6d6035e4 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:58.356622 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf0cc16-5dc7-42b0-985f-5def6d6035e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:58.381107 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-fcf0cc16-5dc7-42b0-985f-5def6d6035e4 None None] MFA Rules not processed for user `87f8cc79586f4cb79c962ede8342f580`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:16:58.382441 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-fcf0cc16-5dc7-42b0-985f-5def6d6035e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:16:58.391071 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 995/1968] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:16:58 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:16:59.003688 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b82b69e-3c1e-490d-a3d7-36865afb22af None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:59.005314 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b82b69e-3c1e-490d-a3d7-36865afb22af None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:59.005847 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b82b69e-3c1e-490d-a3d7-36865afb22af None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:59.012182 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 974/1969] 10.222.0.20 () {54 vars in 921 bytes} [Tue Sep 6 20:16:59 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:16:59.026536 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa12aec0-74b7-48f6-b02c-ca5008989421 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:59.027058 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa12aec0-74b7-48f6-b02c-ca5008989421 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:59.027571 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa12aec0-74b7-48f6-b02c-ca5008989421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:59.055507 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-fa12aec0-74b7-48f6-b02c-ca5008989421 None None] MFA Rules not processed for user `f57872de7bb74ea48d30e93e51830aba`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:16:59.057554 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-fa12aec0-74b7-48f6-b02c-ca5008989421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:16:59.238599 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 996/1970] 10.222.0.20 () {58 vars in 1018 bytes} [Tue Sep 6 20:16:59 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:16:59.270395 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b82b69e-3c1e-490d-a3d7-36865afb22af None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:16:59.275020 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-7b82b69e-3c1e-490d-a3d7-36865afb22af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:16:59.296383 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fcd11d-7aa8-4c6c-a33f-3455b7276166 None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:59.296838 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fcd11d-7aa8-4c6c-a33f-3455b7276166 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:59.297310 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fcd11d-7aa8-4c6c-a33f-3455b7276166 None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:59.306491 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 997/1971] 10.222.0.20 () {54 vars in 961 bytes} [Tue Sep 6 20:16:59 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:16:59.323297 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f7af1c-2766-4b21-aa1a-627ad92d6b14 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:59.323753 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f7af1c-2766-4b21-aa1a-627ad92d6b14 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:59.324675 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f7af1c-2766-4b21-aa1a-627ad92d6b14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:59.383961 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b82b69e-3c1e-490d-a3d7-36865afb22af None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:16:59.406464 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-c8f7af1c-2766-4b21-aa1a-627ad92d6b14 None None] MFA Rules not processed for user `f57872de7bb74ea48d30e93e51830aba`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:16:59.408677 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bc6b77-4105-47da-afe0-ea501e2e66fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:16:59.409765 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bc6b77-4105-47da-afe0-ea501e2e66fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:59.410206 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bc6b77-4105-47da-afe0-ea501e2e66fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:59.410640 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bc6b77-4105-47da-afe0-ea501e2e66fd admin admin] PATH_INFO: `/v3/users/48cca45629ef4e09aac4262616e2792f` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:59.410829 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-c8f7af1c-2766-4b21-aa1a-627ad92d6b14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:16:59.416345 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bc6b77-4105-47da-afe0-ea501e2e66fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=48cca45629ef4e09aac4262616e2792f)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:16:59.419899 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bc6b77-4105-47da-afe0-ea501e2e66fd admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:16:59.433380 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 998/1972] 10.222.0.20 () {58 vars in 1058 bytes} [Tue Sep 6 20:16:59 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:16:59.450340 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f7af1c-2766-4b21-aa1a-627ad92d6b14 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:16:59.472716 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f7af1c-2766-4b21-aa1a-627ad92d6b14 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:16:59.486038 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44094a82-6eda-4b6d-9c43-74948b55da66 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f57872de7bb74ea48d30e93e51830aba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:16:59.487211 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44094a82-6eda-4b6d-9c43-74948b55da66 service placement] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:16:59.488336 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44094a82-6eda-4b6d-9c43-74948b55da66 service placement] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:16:59.489036 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44094a82-6eda-4b6d-9c43-74948b55da66 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:16:59.509829 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44094a82-6eda-4b6d-9c43-74948b55da66 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:16:59.512902 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44094a82-6eda-4b6d-9c43-74948b55da66 service placement] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:16:59.539682 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:16:59.557367 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-d2bc6b77-4105-47da-afe0-ea501e2e66fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48cca45629ef4e09aac4262616e2792f'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:16:59.577943 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-d2bc6b77-4105-47da-afe0-ea501e2e66fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48cca45629ef4e09aac4262616e2792f'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:16:59.582074 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-d2bc6b77-4105-47da-afe0-ea501e2e66fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48cca45629ef4e09aac4262616e2792f'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:16:59.584218 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-d2bc6b77-4105-47da-afe0-ea501e2e66fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48cca45629ef4e09aac4262616e2792f'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:16:59.605232 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-d2bc6b77-4105-47da-afe0-ea501e2e66fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48cca45629ef4e09aac4262616e2792f'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:16:59.620526 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 999/1973] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:16:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 20:16:59.665691 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 975/1974] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:16:59 2022] DELETE /identity/v3/users/48cca45629ef4e09aac4262616e2792f => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:17:00.721887 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44094a82-6eda-4b6d-9c43-74948b55da66 service placement] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:17:00.735258 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44094a82-6eda-4b6d-9c43-74948b55da66 service placement] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:17:00.779186 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:17:00.849995 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e99eba5-6be1-4c41-b145-cb0731dea45b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:17:00.864474 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e99eba5-6be1-4c41-b145-cb0731dea45b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:17:00.865958 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e99eba5-6be1-4c41-b145-cb0731dea45b admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:17:00.867685 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e99eba5-6be1-4c41-b145-cb0731dea45b admin admin] PATH_INFO: `/v3/projects/d3f88491a2ee4a03b15cb8c88a9aaf9b` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:17:00.887367 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e99eba5-6be1-4c41-b145-cb0731dea45b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3f88491a2ee4a03b15cb8c88a9aaf9b)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:17:00.896659 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e99eba5-6be1-4c41-b145-cb0731dea45b admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:17:01.094915 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-0e99eba5-6be1-4c41-b145-cb0731dea45b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3f88491a2ee4a03b15cb8c88a9aaf9b', 'request_id': 'req-0e99eba5-6be1-4c41-b145-cb0731dea45b'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:17:01.111751 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-0e99eba5-6be1-4c41-b145-cb0731dea45b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3f88491a2ee4a03b15cb8c88a9aaf9b', 'request_id': 'req-0e99eba5-6be1-4c41-b145-cb0731dea45b'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:17:01.121175 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-0e99eba5-6be1-4c41-b145-cb0731dea45b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3f88491a2ee4a03b15cb8c88a9aaf9b', 'request_id': 'req-0e99eba5-6be1-4c41-b145-cb0731dea45b'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:17:01.143050 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-0e99eba5-6be1-4c41-b145-cb0731dea45b admin admin] The token cache is being invalidate because project d3f88491a2ee4a03b15cb8c88a9aaf9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81772) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:17:01.143545 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-0e99eba5-6be1-4c41-b145-cb0731dea45b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:17:01.144819 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-0e99eba5-6be1-4c41-b145-cb0731dea45b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:17:01.148594 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1000/1975] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:17:00 2022] DELETE /identity/v3/projects/d3f88491a2ee4a03b15cb8c88a9aaf9b => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:17:01.155034 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af8c7ec-73d5-4989-8107-4858101af55e None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:17:01.155897 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af8c7ec-73d5-4989-8107-4858101af55e None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:17:01.155897 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af8c7ec-73d5-4989-8107-4858101af55e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:17:01.189724 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-6af8c7ec-73d5-4989-8107-4858101af55e None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:17:01.191661 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-6af8c7ec-73d5-4989-8107-4858101af55e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:17:01.291551 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 976/1976] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:17:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:17:01.301514 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168fa693-d166-4ed7-9524-6e32dc997009 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:17:01.301930 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168fa693-d166-4ed7-9524-6e32dc997009 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:17:01.302321 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168fa693-d166-4ed7-9524-6e32dc997009 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:17:01.328592 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-168fa693-d166-4ed7-9524-6e32dc997009 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:17:01.330952 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-168fa693-d166-4ed7-9524-6e32dc997009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:17:01.350682 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1001/1977] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:17:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:17:01.355743 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af8c7ec-73d5-4989-8107-4858101af55e None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:17:01.367261 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af8c7ec-73d5-4989-8107-4858101af55e None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:17:01.372604 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f729f4d-b6ff-42bb-b8ad-7aa514834985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:17:01.373253 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f729f4d-b6ff-42bb-b8ad-7aa514834985 admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:17:01.373531 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f729f4d-b6ff-42bb-b8ad-7aa514834985 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:17:01.373832 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f729f4d-b6ff-42bb-b8ad-7aa514834985 admin admin] PATH_INFO: `/v3/domains` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:17:01.374512 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f729f4d-b6ff-42bb-b8ad-7aa514834985 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:17:01.374809 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f729f4d-b6ff-42bb-b8ad-7aa514834985 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:17:01.377000 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f729f4d-b6ff-42bb-b8ad-7aa514834985 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:17:01.396067 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 977/1978] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 6 20:17:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:17:01.402143 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168fa693-d166-4ed7-9524-6e32dc997009 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:17:01.418855 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168fa693-d166-4ed7-9524-6e32dc997009 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:17:01.426598 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27570817-87fd-4ad8-b5f1-11e09e08dfa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:17:01.427712 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27570817-87fd-4ad8-b5f1-11e09e08dfa0 admin admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:17:01.428205 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27570817-87fd-4ad8-b5f1-11e09e08dfa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:17:01.428648 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27570817-87fd-4ad8-b5f1-11e09e08dfa0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:17:01.430076 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27570817-87fd-4ad8-b5f1-11e09e08dfa0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:17:01.433028 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27570817-87fd-4ad8-b5f1-11e09e08dfa0 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:17:01.486072 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1002/1979] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:17:01 2022] POST /identity/v3/projects => generated 367 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 20:17:01.492217 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f729f4d-b6ff-42bb-b8ad-7aa514834985 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:17:01.517756 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f729f4d-b6ff-42bb-b8ad-7aa514834985 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:17:01.612246 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9be42db-5620-4735-9adb-d69c6104b467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:17:01.613368 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9be42db-5620-4735-9adb-d69c6104b467 admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:17:01.613922 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9be42db-5620-4735-9adb-d69c6104b467 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:17:01.614382 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9be42db-5620-4735-9adb-d69c6104b467 admin admin] PATH_INFO: `/v3/users` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:17:01.615577 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9be42db-5620-4735-9adb-d69c6104b467 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:17:01.619304 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9be42db-5620-4735-9adb-d69c6104b467 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:17:01.660551 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 978/1980] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:17:01 2022] POST /identity/v3/users => generated 284 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:17:01.666989 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27570817-87fd-4ad8-b5f1-11e09e08dfa0 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:17:01.682397 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27570817-87fd-4ad8-b5f1-11e09e08dfa0 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:17:01.689445 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe80884-405d-405d-bf7b-75adc6faa33c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:17:01.690298 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe80884-405d-405d-bf7b-75adc6faa33c admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:17:01.690755 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe80884-405d-405d-bf7b-75adc6faa33c admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:17:01.691212 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe80884-405d-405d-bf7b-75adc6faa33c admin admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:17:01.692039 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe80884-405d-405d-bf7b-75adc6faa33c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:17:01.694367 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe80884-405d-405d-bf7b-75adc6faa33c admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:17:01.708057 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1003/1981] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:17:01 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:17:01.712797 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9be42db-5620-4735-9adb-d69c6104b467 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:17:01.725211 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9be42db-5620-4735-9adb-d69c6104b467 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:17:01.731724 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5888aa-4fe3-4a0b-a19d-bb20d35c4b76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:17:01.732654 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5888aa-4fe3-4a0b-a19d-bb20d35c4b76 admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:17:01.732939 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5888aa-4fe3-4a0b-a19d-bb20d35c4b76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:17:01.733217 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5888aa-4fe3-4a0b-a19d-bb20d35c4b76 admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:17:01.733930 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5888aa-4fe3-4a0b-a19d-bb20d35c4b76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:17:01.736170 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5888aa-4fe3-4a0b-a19d-bb20d35c4b76 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:17:01.749696 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 979/1982] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:17:01 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:17:01.754942 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe80884-405d-405d-bf7b-75adc6faa33c admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:17:01.768905 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe80884-405d-405d-bf7b-75adc6faa33c admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:17:01.775569 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7bd6a7-1a87-4091-b763-b00273b6a185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:17:01.776516 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7bd6a7-1a87-4091-b763-b00273b6a185 admin admin] REQUEST_METHOD: `PUT` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:17:01.776822 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7bd6a7-1a87-4091-b763-b00273b6a185 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:17:01.777666 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7bd6a7-1a87-4091-b763-b00273b6a185 admin admin] PATH_INFO: `/v3/projects/4e56a8c970d04d13ae4c4ce0ebb99bc1/users/e9ef73b591c54751ad30cda81ef4daf3/roles/4eea39e24b06431aa357053fe7907a3d` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:17:01.801303 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7bd6a7-1a87-4091-b763-b00273b6a185 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e56a8c970d04d13ae4c4ce0ebb99bc1, user_id=e9ef73b591c54751ad30cda81ef4daf3, role_id=4eea39e24b06431aa357053fe7907a3d)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:17:01.804405 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7bd6a7-1a87-4091-b763-b00273b6a185 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:17:01.822285 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1004/1983] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:17:01 2022] PUT /identity/v3/projects/4e56a8c970d04d13ae4c4ce0ebb99bc1/users/e9ef73b591c54751ad30cda81ef4daf3/roles/4eea39e24b06431aa357053fe7907a3d => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 20:17:01.839336 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ea03c6-cd51-45b8-bf95-1517bacb6ee6 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:17:01.839819 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ea03c6-cd51-45b8-bf95-1517bacb6ee6 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:17:01.840122 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ea03c6-cd51-45b8-bf95-1517bacb6ee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:17:01.873314 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-60ea03c6-cd51-45b8-bf95-1517bacb6ee6 None None] MFA Rules not processed for user `e9ef73b591c54751ad30cda81ef4daf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:17:01.874651 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-60ea03c6-cd51-45b8-bf95-1517bacb6ee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:17:01.898897 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:17:02.001438 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 980/1984] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:17:01 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:17:02.015948 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7bd6a7-1a87-4091-b763-b00273b6a185 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:17:02.020254 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-cc7bd6a7-1a87-4091-b763-b00273b6a185 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:17:02.115287 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7bd6a7-1a87-4091-b763-b00273b6a185 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:17:02.121142 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81739a41-1e59-4cd8-a444-2089a2f643a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df1bca993bc142908cc7cf85eabd2b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:17:02.121936 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81739a41-1e59-4cd8-a444-2089a2f643a4 service cinder] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:17:02.122267 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81739a41-1e59-4cd8-a444-2089a2f643a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:17:02.122540 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81739a41-1e59-4cd8-a444-2089a2f643a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:17:02.134989 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81739a41-1e59-4cd8-a444-2089a2f643a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:17:02.136599 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81739a41-1e59-4cd8-a444-2089a2f643a4 service cinder] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:17:02.142155 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:17:02.153668 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1005/1985] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:17:02 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:17:04.146630 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63e0080-6e52-4789-802e-c54ccf2b2b4d None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:17:04.147545 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63e0080-6e52-4789-802e-c54ccf2b2b4d None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:17:04.147545 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63e0080-6e52-4789-802e-c54ccf2b2b4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:17:04.186013 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-e63e0080-6e52-4789-802e-c54ccf2b2b4d None None] MFA Rules not processed for user `861d88f0f5674b299575ad2e81dbaaf6`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:17:04.188993 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-e63e0080-6e52-4789-802e-c54ccf2b2b4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:17:04.303968 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 981/1986] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:17:04 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:17:12.160859 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=1 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:17:13.898865 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81739a41-1e59-4cd8-a444-2089a2f643a4 service cinder] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:17:14.045944 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81739a41-1e59-4cd8-a444-2089a2f643a4 service cinder] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:17:14.065176 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e43e4c-7976-4ce6-ab08-90bf402bafce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:17:14.066356 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e43e4c-7976-4ce6-ab08-90bf402bafce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:17:14.076564 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e43e4c-7976-4ce6-ab08-90bf402bafce admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:17:14.076564 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e43e4c-7976-4ce6-ab08-90bf402bafce admin admin] PATH_INFO: `/v3/users/e9ef73b591c54751ad30cda81ef4daf3` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:17:14.076564 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e43e4c-7976-4ce6-ab08-90bf402bafce admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9ef73b591c54751ad30cda81ef4daf3)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:17:14.076564 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e43e4c-7976-4ce6-ab08-90bf402bafce admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:17:14.177781 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-a0e43e4c-7976-4ce6-ab08-90bf402bafce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9ef73b591c54751ad30cda81ef4daf3'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:17:14.179129 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-a0e43e4c-7976-4ce6-ab08-90bf402bafce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9ef73b591c54751ad30cda81ef4daf3'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:17:14.207734 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-a0e43e4c-7976-4ce6-ab08-90bf402bafce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9ef73b591c54751ad30cda81ef4daf3'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:17:14.208676 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-a0e43e4c-7976-4ce6-ab08-90bf402bafce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9ef73b591c54751ad30cda81ef4daf3'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:17:14.219696 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-a0e43e4c-7976-4ce6-ab08-90bf402bafce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9ef73b591c54751ad30cda81ef4daf3'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:17:14.239733 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1006/1987] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:17:13 2022] DELETE /identity/v3/users/e9ef73b591c54751ad30cda81ef4daf3 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:17:14.244752 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4420d0-ef41-4f23-96b9-00fb342d2f46 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:17:14.245402 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4420d0-ef41-4f23-96b9-00fb342d2f46 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:17:14.245402 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4420d0-ef41-4f23-96b9-00fb342d2f46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:17:14.268471 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-2d4420d0-ef41-4f23-96b9-00fb342d2f46 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:17:14.269916 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-2d4420d0-ef41-4f23-96b9-00fb342d2f46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:17:14.347856 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 982/1988] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:17:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:17:14.358711 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e43e4c-7976-4ce6-ab08-90bf402bafce admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:17:14.360303 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-a0e43e4c-7976-4ce6-ab08-90bf402bafce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:17:14.430546 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e43e4c-7976-4ce6-ab08-90bf402bafce admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:17:14.434625 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c3e545-e24a-4e19-8f66-7b3feb653921 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:17:14.435263 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c3e545-e24a-4e19-8f66-7b3feb653921 service neutron] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:17:14.435550 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c3e545-e24a-4e19-8f66-7b3feb653921 service neutron] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:17:14.435827 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c3e545-e24a-4e19-8f66-7b3feb653921 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:17:14.446733 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c3e545-e24a-4e19-8f66-7b3feb653921 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:17:14.448237 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c3e545-e24a-4e19-8f66-7b3feb653921 service neutron] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:17:14.460148 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1007/1989] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:17:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:17:14.852746 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4420d0-ef41-4f23-96b9-00fb342d2f46 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:17:14.857703 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-2d4420d0-ef41-4f23-96b9-00fb342d2f46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:17:14.914954 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4420d0-ef41-4f23-96b9-00fb342d2f46 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:17:14.928270 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cdd93a-7dfd-42e5-9cbb-5a15d9325ff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:17:14.930587 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cdd93a-7dfd-42e5-9cbb-5a15d9325ff8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:17:14.930587 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cdd93a-7dfd-42e5-9cbb-5a15d9325ff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:17:14.930587 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cdd93a-7dfd-42e5-9cbb-5a15d9325ff8 admin admin] PATH_INFO: `/v3/projects/4e56a8c970d04d13ae4c4ce0ebb99bc1` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:17:14.932146 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cdd93a-7dfd-42e5-9cbb-5a15d9325ff8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e56a8c970d04d13ae4c4ce0ebb99bc1)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:17:14.934731 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cdd93a-7dfd-42e5-9cbb-5a15d9325ff8 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:17:15.028060 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-42cdd93a-7dfd-42e5-9cbb-5a15d9325ff8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e56a8c970d04d13ae4c4ce0ebb99bc1', 'request_id': 'req-42cdd93a-7dfd-42e5-9cbb-5a15d9325ff8'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:17:15.033371 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:17:15.042579 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-42cdd93a-7dfd-42e5-9cbb-5a15d9325ff8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e56a8c970d04d13ae4c4ce0ebb99bc1', 'request_id': 'req-42cdd93a-7dfd-42e5-9cbb-5a15d9325ff8'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:17:15.062205 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-42cdd93a-7dfd-42e5-9cbb-5a15d9325ff8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e56a8c970d04d13ae4c4ce0ebb99bc1', 'request_id': 'req-42cdd93a-7dfd-42e5-9cbb-5a15d9325ff8'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:17:15.083590 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-42cdd93a-7dfd-42e5-9cbb-5a15d9325ff8 admin admin] The token cache is being invalidate because project 4e56a8c970d04d13ae4c4ce0ebb99bc1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81771) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:17:15.083590 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-42cdd93a-7dfd-42e5-9cbb-5a15d9325ff8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:17:15.084267 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-42cdd93a-7dfd-42e5-9cbb-5a15d9325ff8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:17:15.093121 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 983/1990] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:17:14 2022] DELETE /identity/v3/projects/4e56a8c970d04d13ae4c4ce0ebb99bc1 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:17:15.100071 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6273203-24e9-42ac-8309-23cb9719899a None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:17:15.100708 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6273203-24e9-42ac-8309-23cb9719899a None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:17:15.100917 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6273203-24e9-42ac-8309-23cb9719899a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:17:15.143135 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-c6273203-24e9-42ac-8309-23cb9719899a None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:17:15.148515 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-c6273203-24e9-42ac-8309-23cb9719899a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:17:15.175899 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:17:15.262980 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1008/1991] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:17:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:17:15.277104 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79fbde6-0c98-48a6-a3f7-e50d05ee32a8 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:17:15.277980 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79fbde6-0c98-48a6-a3f7-e50d05ee32a8 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:17:15.277980 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79fbde6-0c98-48a6-a3f7-e50d05ee32a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:17:15.309324 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-d79fbde6-0c98-48a6-a3f7-e50d05ee32a8 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:17:15.311065 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-d79fbde6-0c98-48a6-a3f7-e50d05ee32a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:17:15.323991 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 984/1992] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:17:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:17:15.329460 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6273203-24e9-42ac-8309-23cb9719899a None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:17:15.341595 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6273203-24e9-42ac-8309-23cb9719899a None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:17:15.349604 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7f042c-6481-4dec-9933-33fbad43694c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:17:15.350551 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7f042c-6481-4dec-9933-33fbad43694c admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:17:15.351066 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7f042c-6481-4dec-9933-33fbad43694c admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:17:15.351514 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7f042c-6481-4dec-9933-33fbad43694c admin admin] PATH_INFO: `/v3/domains` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:17:15.352478 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7f042c-6481-4dec-9933-33fbad43694c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:17:15.353125 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7f042c-6481-4dec-9933-33fbad43694c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:17:15.355739 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7f042c-6481-4dec-9933-33fbad43694c admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:17:15.382172 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1009/1993] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 6 20:17:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:17:15.383586 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79fbde6-0c98-48a6-a3f7-e50d05ee32a8 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:17:15.409522 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79fbde6-0c98-48a6-a3f7-e50d05ee32a8 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:17:15.419013 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2bf527-02a1-4753-8914-1533811d28c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:17:15.420363 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2bf527-02a1-4753-8914-1533811d28c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:17:15.420864 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2bf527-02a1-4753-8914-1533811d28c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:17:15.423163 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2bf527-02a1-4753-8914-1533811d28c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:17:15.424627 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2bf527-02a1-4753-8914-1533811d28c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:17:15.429142 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2bf527-02a1-4753-8914-1533811d28c8 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:17:15.513320 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 985/1994] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:17:15 2022] POST /identity/v3/projects => generated 405 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 20:17:15.519305 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7f042c-6481-4dec-9933-33fbad43694c admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:17:15.532210 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7f042c-6481-4dec-9933-33fbad43694c admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:17:15.652631 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eba925-1450-436b-b2cf-a915a0f154c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:17:15.653575 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eba925-1450-436b-b2cf-a915a0f154c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:17:15.654056 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eba925-1450-436b-b2cf-a915a0f154c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:17:15.654632 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eba925-1450-436b-b2cf-a915a0f154c9 admin admin] PATH_INFO: `/v3/users` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:17:15.655509 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2eba925-1450-436b-b2cf-a915a0f154c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:17:15.658328 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2eba925-1450-436b-b2cf-a915a0f154c9 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:17:15.708474 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1010/1995] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:17:15 2022] POST /identity/v3/users => generated 303 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:17:15.737526 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2bf527-02a1-4753-8914-1533811d28c8 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:17:15.751093 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2bf527-02a1-4753-8914-1533811d28c8 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:17:15.759507 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ec3875-ce6c-41c6-975c-34d7c83ff23b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:17:15.759507 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ec3875-ce6c-41c6-975c-34d7c83ff23b admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:17:15.759507 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ec3875-ce6c-41c6-975c-34d7c83ff23b admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:17:15.760073 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ec3875-ce6c-41c6-975c-34d7c83ff23b admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:17:15.760611 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ec3875-ce6c-41c6-975c-34d7c83ff23b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:17:15.762472 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ec3875-ce6c-41c6-975c-34d7c83ff23b admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:17:15.784637 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 986/1996] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:17:15 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:17:15.790465 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eba925-1450-436b-b2cf-a915a0f154c9 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:17:15.803387 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eba925-1450-436b-b2cf-a915a0f154c9 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:17:15.813582 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd8a593-4f5c-4c9c-9adf-a85a7c30efaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:17:15.813582 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd8a593-4f5c-4c9c-9adf-a85a7c30efaf admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:17:15.813582 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd8a593-4f5c-4c9c-9adf-a85a7c30efaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:17:15.813582 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd8a593-4f5c-4c9c-9adf-a85a7c30efaf admin admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:17:15.814358 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd8a593-4f5c-4c9c-9adf-a85a7c30efaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:17:15.815136 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd8a593-4f5c-4c9c-9adf-a85a7c30efaf admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:17:15.829162 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1011/1997] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:17:15 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:17:15.836123 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ec3875-ce6c-41c6-975c-34d7c83ff23b admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:17:15.855462 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ec3875-ce6c-41c6-975c-34d7c83ff23b admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:17:15.864196 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4552e0d9-f121-486c-880f-1d56c2269111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:17:15.865431 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4552e0d9-f121-486c-880f-1d56c2269111 admin admin] REQUEST_METHOD: `PUT` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:17:15.866001 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4552e0d9-f121-486c-880f-1d56c2269111 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:17:15.866518 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4552e0d9-f121-486c-880f-1d56c2269111 admin admin] PATH_INFO: `/v3/projects/5aa6a70b05584269b5c2cb9ff255731b/users/2b2a39a34cb0467885dcf86c543934ad/roles/4eea39e24b06431aa357053fe7907a3d` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:17:15.889906 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4552e0d9-f121-486c-880f-1d56c2269111 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5aa6a70b05584269b5c2cb9ff255731b, user_id=2b2a39a34cb0467885dcf86c543934ad, role_id=4eea39e24b06431aa357053fe7907a3d)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:17:15.893316 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4552e0d9-f121-486c-880f-1d56c2269111 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:17:15.920665 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 987/1998] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:17:15 2022] PUT /identity/v3/projects/5aa6a70b05584269b5c2cb9ff255731b/users/2b2a39a34cb0467885dcf86c543934ad/roles/4eea39e24b06431aa357053fe7907a3d => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:17:15.927281 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da02e8f3-47cb-4bcc-9ba5-bdc88d21c540 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:17:15.927725 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da02e8f3-47cb-4bcc-9ba5-bdc88d21c540 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:17:15.928144 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da02e8f3-47cb-4bcc-9ba5-bdc88d21c540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:17:15.963573 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-da02e8f3-47cb-4bcc-9ba5-bdc88d21c540 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:17:15.965797 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-da02e8f3-47cb-4bcc-9ba5-bdc88d21c540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:17:16.055742 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1012/1999] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:17:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:17:16.067168 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4552e0d9-f121-486c-880f-1d56c2269111 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:17:16.070116 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-4552e0d9-f121-486c-880f-1d56c2269111 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:17:16.188191 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4552e0d9-f121-486c-880f-1d56c2269111 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:17:16.196384 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5be3d25-0ad9-44dd-8869-8d723671e1ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:17:16.197580 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5be3d25-0ad9-44dd-8869-8d723671e1ac service neutron] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:17:16.198203 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5be3d25-0ad9-44dd-8869-8d723671e1ac service neutron] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:17:16.198779 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5be3d25-0ad9-44dd-8869-8d723671e1ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:17:16.216937 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5be3d25-0ad9-44dd-8869-8d723671e1ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:17:16.218467 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5be3d25-0ad9-44dd-8869-8d723671e1ac service neutron] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:17:16.233424 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 988/2000] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:17:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 20:17:17.525172 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869c4917-fa01-4c4e-9267-44f2059ddb18 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:17:17.526046 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869c4917-fa01-4c4e-9267-44f2059ddb18 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:17:17.526345 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869c4917-fa01-4c4e-9267-44f2059ddb18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:17:17.576489 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-869c4917-fa01-4c4e-9267-44f2059ddb18 None None] MFA Rules not processed for user `2b2a39a34cb0467885dcf86c543934ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:17:17.578270 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-869c4917-fa01-4c4e-9267-44f2059ddb18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:17:17.612398 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:17:17.713137 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1013/2001] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:17:17 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:17:17.722770 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5be3d25-0ad9-44dd-8869-8d723671e1ac service neutron] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:17:17.742709 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5be3d25-0ad9-44dd-8869-8d723671e1ac service neutron] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:17:17.751150 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a2dbfb-48fd-4066-a8a0-12219b30bca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:17:17.752213 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a2dbfb-48fd-4066-a8a0-12219b30bca0 admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:17:17.752702 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a2dbfb-48fd-4066-a8a0-12219b30bca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:17:17.753342 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a2dbfb-48fd-4066-a8a0-12219b30bca0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:17:17.757088 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a2dbfb-48fd-4066-a8a0-12219b30bca0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:17:17.758918 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a2dbfb-48fd-4066-a8a0-12219b30bca0 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:17:17.810980 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 989/2002] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:17:17 2022] POST /identity/v3/projects => generated 405 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:17:17.816041 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869c4917-fa01-4c4e-9267-44f2059ddb18 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:17:17.830552 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869c4917-fa01-4c4e-9267-44f2059ddb18 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:17:17.945501 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3938b0a7-21e8-4c47-89f9-2a8506a47852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:17:17.946300 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3938b0a7-21e8-4c47-89f9-2a8506a47852 admin admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:17:17.946872 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3938b0a7-21e8-4c47-89f9-2a8506a47852 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:17:17.947470 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3938b0a7-21e8-4c47-89f9-2a8506a47852 admin admin] PATH_INFO: `/v3/users` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:17:17.948958 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3938b0a7-21e8-4c47-89f9-2a8506a47852 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:17:17.954305 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3938b0a7-21e8-4c47-89f9-2a8506a47852 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:17:18.003453 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1014/2003] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:17:17 2022] POST /identity/v3/users => generated 309 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 20:17:18.008738 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a2dbfb-48fd-4066-a8a0-12219b30bca0 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:17:18.023849 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a2dbfb-48fd-4066-a8a0-12219b30bca0 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:17:18.030308 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dc18fd-160c-424b-803f-f5971ea2b759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:17:18.031187 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dc18fd-160c-424b-803f-f5971ea2b759 admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:17:18.032119 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dc18fd-160c-424b-803f-f5971ea2b759 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:17:18.032789 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dc18fd-160c-424b-803f-f5971ea2b759 admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:17:18.033791 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dc18fd-160c-424b-803f-f5971ea2b759 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:17:18.035857 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dc18fd-160c-424b-803f-f5971ea2b759 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:17:18.051342 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 990/2004] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:17:18 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 20:17:18.058356 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3938b0a7-21e8-4c47-89f9-2a8506a47852 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:17:18.071676 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3938b0a7-21e8-4c47-89f9-2a8506a47852 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:17:18.077037 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab53060-9c59-4abe-9737-e02455f68400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:17:18.077800 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab53060-9c59-4abe-9737-e02455f68400 admin admin] REQUEST_METHOD: `PUT` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:17:18.078277 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab53060-9c59-4abe-9737-e02455f68400 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:17:18.078732 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab53060-9c59-4abe-9737-e02455f68400 admin admin] PATH_INFO: `/v3/projects/284ccbe47d3f4a408f9c2f375a1e622f/users/ae81f713c1cc4d379c83385aaefc9510/roles/e0ab41d8ae22474688b5361bcc75e0f0` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:17:18.099477 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab53060-9c59-4abe-9737-e02455f68400 admin admin] RBAC: Authorizing `identity:create_grant(project_id=284ccbe47d3f4a408f9c2f375a1e622f, user_id=ae81f713c1cc4d379c83385aaefc9510, role_id=e0ab41d8ae22474688b5361bcc75e0f0)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:17:18.102437 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab53060-9c59-4abe-9737-e02455f68400 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:17:18.121685 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1015/2005] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:17:18 2022] PUT /identity/v3/projects/284ccbe47d3f4a408f9c2f375a1e622f/users/ae81f713c1cc4d379c83385aaefc9510/roles/e0ab41d8ae22474688b5361bcc75e0f0 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:17:18.127951 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dc18fd-160c-424b-803f-f5971ea2b759 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:17:18.145414 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dc18fd-160c-424b-803f-f5971ea2b759 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:17:18.227161 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:17:18.262319 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9fc28d-d4b2-442d-9156-80287bd633fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:17:18.263472 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9fc28d-d4b2-442d-9156-80287bd633fa admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:17:18.264109 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9fc28d-d4b2-442d-9156-80287bd633fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:17:18.264517 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9fc28d-d4b2-442d-9156-80287bd633fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:17:18.266367 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9fc28d-d4b2-442d-9156-80287bd633fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:17:18.269477 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9fc28d-d4b2-442d-9156-80287bd633fa admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:17:18.298157 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 991/2006] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:17:18 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:17:18.305253 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab53060-9c59-4abe-9737-e02455f68400 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:17:18.327754 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab53060-9c59-4abe-9737-e02455f68400 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:17:18.338971 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f76bce-ecb3-4524-b2ea-8dd2325fc1b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:17:18.340037 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f76bce-ecb3-4524-b2ea-8dd2325fc1b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:17:18.340481 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f76bce-ecb3-4524-b2ea-8dd2325fc1b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:17:18.340942 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f76bce-ecb3-4524-b2ea-8dd2325fc1b9 admin admin] PATH_INFO: `/v3/projects/284ccbe47d3f4a408f9c2f375a1e622f/users/ae81f713c1cc4d379c83385aaefc9510/roles/e0ab41d8ae22474688b5361bcc75e0f0` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:17:18.344985 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f76bce-ecb3-4524-b2ea-8dd2325fc1b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=284ccbe47d3f4a408f9c2f375a1e622f, user_id=ae81f713c1cc4d379c83385aaefc9510, role_id=e0ab41d8ae22474688b5361bcc75e0f0)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:17:18.350504 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f76bce-ecb3-4524-b2ea-8dd2325fc1b9 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:17:18.373274 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1016/2007] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:17:18 2022] PUT /identity/v3/projects/284ccbe47d3f4a408f9c2f375a1e622f/users/ae81f713c1cc4d379c83385aaefc9510/roles/e0ab41d8ae22474688b5361bcc75e0f0 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:17:19.628684 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c077e4-e4fa-4d69-852f-f07acc22280d None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:17:19.629495 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c077e4-e4fa-4d69-852f-f07acc22280d None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:17:19.630006 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c077e4-e4fa-4d69-852f-f07acc22280d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:17:19.788858 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-03c077e4-e4fa-4d69-852f-f07acc22280d None None] MFA Rules not processed for user `ae81f713c1cc4d379c83385aaefc9510`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:17:19.791543 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-03c077e4-e4fa-4d69-852f-f07acc22280d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:17:19.977976 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 992/2008] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:17:19 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 348 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:17:20.763372 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63522512-7c63-4add-bf47-b8685da60179 None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:17:20.764280 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63522512-7c63-4add-bf47-b8685da60179 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:17:20.764280 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63522512-7c63-4add-bf47-b8685da60179 None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:17:20.767741 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1017/2009] 10.222.0.20 () {54 vars in 921 bytes} [Tue Sep 6 20:17:20 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:17:20.788449 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2740244-0311-4fb1-88aa-e77dc655862d None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:17:20.789005 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2740244-0311-4fb1-88aa-e77dc655862d None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:17:20.789828 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2740244-0311-4fb1-88aa-e77dc655862d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:17:20.833066 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-c2740244-0311-4fb1-88aa-e77dc655862d None None] MFA Rules not processed for user `f57872de7bb74ea48d30e93e51830aba`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:17:20.835545 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-c2740244-0311-4fb1-88aa-e77dc655862d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:17:20.997082 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 993/2010] 10.222.0.20 () {58 vars in 1018 bytes} [Tue Sep 6 20:17:20 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:17:21.015306 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63522512-7c63-4add-bf47-b8685da60179 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:17:21.017531 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-63522512-7c63-4add-bf47-b8685da60179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:17:21.054781 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63522512-7c63-4add-bf47-b8685da60179 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:17:21.067741 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7459e13b-e303-4615-87f3-6d472ce43d5c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f57872de7bb74ea48d30e93e51830aba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:17:21.068804 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7459e13b-e303-4615-87f3-6d472ce43d5c service placement] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:17:21.069544 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7459e13b-e303-4615-87f3-6d472ce43d5c service placement] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:17:21.069890 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7459e13b-e303-4615-87f3-6d472ce43d5c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:17:21.092901 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7459e13b-e303-4615-87f3-6d472ce43d5c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:17:21.096973 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7459e13b-e303-4615-87f3-6d472ce43d5c service placement] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:17:21.120132 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1018/2011] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:17:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 20:17:22.143839 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2740244-0311-4fb1-88aa-e77dc655862d None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:17:22.236215 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2740244-0311-4fb1-88aa-e77dc655862d None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:17:22.242187 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904de5dc-cd08-4d2c-a890-de62ba3ae0f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:17:22.242916 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904de5dc-cd08-4d2c-a890-de62ba3ae0f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:17:22.243125 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904de5dc-cd08-4d2c-a890-de62ba3ae0f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:17:22.243436 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904de5dc-cd08-4d2c-a890-de62ba3ae0f7 admin admin] PATH_INFO: `/v3/users/2b2a39a34cb0467885dcf86c543934ad` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:17:22.244894 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904de5dc-cd08-4d2c-a890-de62ba3ae0f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b2a39a34cb0467885dcf86c543934ad)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:17:22.246980 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904de5dc-cd08-4d2c-a890-de62ba3ae0f7 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:17:22.253343 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:17:22.335850 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-904de5dc-cd08-4d2c-a890-de62ba3ae0f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b2a39a34cb0467885dcf86c543934ad'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:17:22.351169 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-904de5dc-cd08-4d2c-a890-de62ba3ae0f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b2a39a34cb0467885dcf86c543934ad'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:17:22.352319 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-904de5dc-cd08-4d2c-a890-de62ba3ae0f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b2a39a34cb0467885dcf86c543934ad'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:17:22.353380 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-904de5dc-cd08-4d2c-a890-de62ba3ae0f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b2a39a34cb0467885dcf86c543934ad'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:17:22.374430 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-904de5dc-cd08-4d2c-a890-de62ba3ae0f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b2a39a34cb0467885dcf86c543934ad'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:17:22.415418 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 994/2012] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:17:22 2022] DELETE /identity/v3/users/2b2a39a34cb0467885dcf86c543934ad => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:17:22.768811 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7459e13b-e303-4615-87f3-6d472ce43d5c service placement] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:17:22.796461 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7459e13b-e303-4615-87f3-6d472ce43d5c service placement] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:17:22.883418 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:17:22.895588 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dfd11f-2758-4361-9764-a9bd0e169119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:17:22.897109 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dfd11f-2758-4361-9764-a9bd0e169119 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:17:22.897604 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dfd11f-2758-4361-9764-a9bd0e169119 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:17:22.898492 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dfd11f-2758-4361-9764-a9bd0e169119 admin admin] PATH_INFO: `/v3/projects/5aa6a70b05584269b5c2cb9ff255731b` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:17:22.901218 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dfd11f-2758-4361-9764-a9bd0e169119 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5aa6a70b05584269b5c2cb9ff255731b)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:17:22.905124 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dfd11f-2758-4361-9764-a9bd0e169119 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:17:23.020791 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-44dfd11f-2758-4361-9764-a9bd0e169119 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5aa6a70b05584269b5c2cb9ff255731b', 'request_id': 'req-44dfd11f-2758-4361-9764-a9bd0e169119'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:17:23.043196 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-44dfd11f-2758-4361-9764-a9bd0e169119 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5aa6a70b05584269b5c2cb9ff255731b', 'request_id': 'req-44dfd11f-2758-4361-9764-a9bd0e169119'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:17:23.053487 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-44dfd11f-2758-4361-9764-a9bd0e169119 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5aa6a70b05584269b5c2cb9ff255731b', 'request_id': 'req-44dfd11f-2758-4361-9764-a9bd0e169119'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:17:23.070247 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-44dfd11f-2758-4361-9764-a9bd0e169119 admin admin] The token cache is being invalidate because project 5aa6a70b05584269b5c2cb9ff255731b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81772) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:17:23.070681 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-44dfd11f-2758-4361-9764-a9bd0e169119 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:17:23.071840 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-44dfd11f-2758-4361-9764-a9bd0e169119 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:17:23.077738 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1019/2013] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:17:22 2022] DELETE /identity/v3/projects/5aa6a70b05584269b5c2cb9ff255731b => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:17:23.080809 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904de5dc-cd08-4d2c-a890-de62ba3ae0f7 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:17:23.082628 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-904de5dc-cd08-4d2c-a890-de62ba3ae0f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:17:23.101352 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904de5dc-cd08-4d2c-a890-de62ba3ae0f7 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:17:23.214161 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8de781-ee54-417a-abae-841d34a25b45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:17:23.215429 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8de781-ee54-417a-abae-841d34a25b45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:17:23.215887 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8de781-ee54-417a-abae-841d34a25b45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:17:23.216315 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8de781-ee54-417a-abae-841d34a25b45 admin admin] PATH_INFO: `/v3/users/ae81f713c1cc4d379c83385aaefc9510` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:17:23.217929 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8de781-ee54-417a-abae-841d34a25b45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae81f713c1cc4d379c83385aaefc9510)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:17:23.221029 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8de781-ee54-417a-abae-841d34a25b45 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:17:23.307111 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-4f8de781-ee54-417a-abae-841d34a25b45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae81f713c1cc4d379c83385aaefc9510'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:17:23.320663 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-4f8de781-ee54-417a-abae-841d34a25b45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae81f713c1cc4d379c83385aaefc9510'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:17:23.321766 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-4f8de781-ee54-417a-abae-841d34a25b45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae81f713c1cc4d379c83385aaefc9510'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:17:23.322812 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-4f8de781-ee54-417a-abae-841d34a25b45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae81f713c1cc4d379c83385aaefc9510'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:17:23.337782 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-4f8de781-ee54-417a-abae-841d34a25b45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae81f713c1cc4d379c83385aaefc9510'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:17:23.367174 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 995/2014] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:17:23 2022] DELETE /identity/v3/users/ae81f713c1cc4d379c83385aaefc9510 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 20:17:23.734159 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dfd11f-2758-4361-9764-a9bd0e169119 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:17:23.740328 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-44dfd11f-2758-4361-9764-a9bd0e169119 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:17:23.771592 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dfd11f-2758-4361-9764-a9bd0e169119 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:17:23.906177 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5756d980-f876-4791-84b7-f95055b53812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:17:23.908665 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5756d980-f876-4791-84b7-f95055b53812 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:17:23.909099 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5756d980-f876-4791-84b7-f95055b53812 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:17:23.909655 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5756d980-f876-4791-84b7-f95055b53812 admin admin] PATH_INFO: `/v3/projects/284ccbe47d3f4a408f9c2f375a1e622f` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:17:23.912026 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5756d980-f876-4791-84b7-f95055b53812 admin admin] RBAC: Authorizing `identity:delete_project(project_id=284ccbe47d3f4a408f9c2f375a1e622f)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:17:23.916314 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5756d980-f876-4791-84b7-f95055b53812 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:17:24.015407 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-5756d980-f876-4791-84b7-f95055b53812 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '284ccbe47d3f4a408f9c2f375a1e622f', 'request_id': 'req-5756d980-f876-4791-84b7-f95055b53812'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:17:24.033026 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-5756d980-f876-4791-84b7-f95055b53812 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '284ccbe47d3f4a408f9c2f375a1e622f', 'request_id': 'req-5756d980-f876-4791-84b7-f95055b53812'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:17:24.049198 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-5756d980-f876-4791-84b7-f95055b53812 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '284ccbe47d3f4a408f9c2f375a1e622f', 'request_id': 'req-5756d980-f876-4791-84b7-f95055b53812'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:17:24.078797 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-5756d980-f876-4791-84b7-f95055b53812 admin admin] The token cache is being invalidate because project 284ccbe47d3f4a408f9c2f375a1e622f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81772) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:17:24.079291 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-5756d980-f876-4791-84b7-f95055b53812 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:17:24.081059 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-5756d980-f876-4791-84b7-f95055b53812 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:17:24.084072 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1020/2015] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:17:23 2022] DELETE /identity/v3/projects/284ccbe47d3f4a408f9c2f375a1e622f => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:17:24.092476 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7655025e-9342-4e01-be83-4fd652ac8ef1 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:17:24.092979 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7655025e-9342-4e01-be83-4fd652ac8ef1 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:17:24.093468 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7655025e-9342-4e01-be83-4fd652ac8ef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:17:24.108222 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:17:24.151455 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-7655025e-9342-4e01-be83-4fd652ac8ef1 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:17:24.152976 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-7655025e-9342-4e01-be83-4fd652ac8ef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:17:24.249128 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 996/2016] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:17:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:17:24.264117 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46608ee8-2cf7-4e33-92bc-18ff76626252 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:17:24.264616 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46608ee8-2cf7-4e33-92bc-18ff76626252 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:17:24.265145 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46608ee8-2cf7-4e33-92bc-18ff76626252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:17:24.305475 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-46608ee8-2cf7-4e33-92bc-18ff76626252 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:17:24.306890 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-46608ee8-2cf7-4e33-92bc-18ff76626252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:17:24.316342 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1021/2017] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:17:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:17:24.324953 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7655025e-9342-4e01-be83-4fd652ac8ef1 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:17:24.361991 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7655025e-9342-4e01-be83-4fd652ac8ef1 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:17:24.370678 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44bcabb-deea-45ed-9f64-62e1f4ce52a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:17:24.371774 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44bcabb-deea-45ed-9f64-62e1f4ce52a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:17:24.372215 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44bcabb-deea-45ed-9f64-62e1f4ce52a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:17:24.372672 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44bcabb-deea-45ed-9f64-62e1f4ce52a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:17:24.373821 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44bcabb-deea-45ed-9f64-62e1f4ce52a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:17:24.374339 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44bcabb-deea-45ed-9f64-62e1f4ce52a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:17:24.377857 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44bcabb-deea-45ed-9f64-62e1f4ce52a4 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:17:24.400753 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 997/2018] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 6 20:17:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:17:24.407689 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46608ee8-2cf7-4e33-92bc-18ff76626252 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:17:24.421172 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46608ee8-2cf7-4e33-92bc-18ff76626252 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:17:24.430850 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464d8490-3c49-422d-8d61-cd3c81e887bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:17:24.431602 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464d8490-3c49-422d-8d61-cd3c81e887bf admin admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:17:24.431950 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464d8490-3c49-422d-8d61-cd3c81e887bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:17:24.432209 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464d8490-3c49-422d-8d61-cd3c81e887bf admin admin] PATH_INFO: `/v3/projects` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:17:24.433038 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464d8490-3c49-422d-8d61-cd3c81e887bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:17:24.435479 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464d8490-3c49-422d-8d61-cd3c81e887bf admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:17:24.460306 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:17:24.504154 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1022/2019] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:17:24 2022] POST /identity/v3/projects => generated 371 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:17:24.510881 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44bcabb-deea-45ed-9f64-62e1f4ce52a4 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:17:24.529996 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44bcabb-deea-45ed-9f64-62e1f4ce52a4 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:17:24.672649 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b27f69-64fb-4c23-9c60-407a4c805e23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:17:24.673679 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b27f69-64fb-4c23-9c60-407a4c805e23 admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:17:24.673960 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b27f69-64fb-4c23-9c60-407a4c805e23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:17:24.674251 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b27f69-64fb-4c23-9c60-407a4c805e23 admin admin] PATH_INFO: `/v3/users` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:17:24.674918 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b27f69-64fb-4c23-9c60-407a4c805e23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:17:24.678278 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b27f69-64fb-4c23-9c60-407a4c805e23 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:17:24.709337 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 998/2020] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:17:24 2022] POST /identity/v3/users => generated 286 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:17:24.715393 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464d8490-3c49-422d-8d61-cd3c81e887bf admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:17:24.727757 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464d8490-3c49-422d-8d61-cd3c81e887bf admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:17:24.733298 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86182b80-8fb3-4343-8ccf-56722b5ba40d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:17:24.734294 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86182b80-8fb3-4343-8ccf-56722b5ba40d admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:17:24.735060 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86182b80-8fb3-4343-8ccf-56722b5ba40d admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:17:24.735567 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86182b80-8fb3-4343-8ccf-56722b5ba40d admin admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:17:24.736301 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86182b80-8fb3-4343-8ccf-56722b5ba40d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:17:24.739668 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86182b80-8fb3-4343-8ccf-56722b5ba40d admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:17:24.790726 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1023/2021] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:17:24 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 20:17:24.798830 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b27f69-64fb-4c23-9c60-407a4c805e23 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:17:24.820070 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b27f69-64fb-4c23-9c60-407a4c805e23 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:17:24.827226 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11054d6c-79b4-4747-940f-516159ffd3f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:17:24.828212 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11054d6c-79b4-4747-940f-516159ffd3f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:17:24.828883 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11054d6c-79b4-4747-940f-516159ffd3f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:17:24.829565 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11054d6c-79b4-4747-940f-516159ffd3f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:17:24.830493 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11054d6c-79b4-4747-940f-516159ffd3f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:17:24.833216 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11054d6c-79b4-4747-940f-516159ffd3f8 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:17:24.852916 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 999/2022] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:17:24 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:17:24.858586 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86182b80-8fb3-4343-8ccf-56722b5ba40d admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:17:24.889218 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86182b80-8fb3-4343-8ccf-56722b5ba40d admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:17:24.907984 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b13b460-f4bc-4d4e-8836-a23ccc4696a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:17:24.910058 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b13b460-f4bc-4d4e-8836-a23ccc4696a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:17:24.910742 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b13b460-f4bc-4d4e-8836-a23ccc4696a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:17:24.911245 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b13b460-f4bc-4d4e-8836-a23ccc4696a6 admin admin] PATH_INFO: `/v3/projects/f87426bf9f944b258c1c80097faeb0c7/users/241c74785dc14bfcae825157e32ba0a4/roles/4eea39e24b06431aa357053fe7907a3d` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:17:24.944096 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b13b460-f4bc-4d4e-8836-a23ccc4696a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f87426bf9f944b258c1c80097faeb0c7, user_id=241c74785dc14bfcae825157e32ba0a4, role_id=4eea39e24b06431aa357053fe7907a3d)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:17:24.950255 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b13b460-f4bc-4d4e-8836-a23ccc4696a6 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:17:24.983904 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1024/2023] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:17:24 2022] PUT /identity/v3/projects/f87426bf9f944b258c1c80097faeb0c7/users/241c74785dc14bfcae825157e32ba0a4/roles/4eea39e24b06431aa357053fe7907a3d => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 20:17:24.998189 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f2d54f-900d-4ca9-b818-722fce4819f8 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:17:24.998622 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f2d54f-900d-4ca9-b818-722fce4819f8 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:17:24.999018 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f2d54f-900d-4ca9-b818-722fce4819f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:17:25.043436 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-72f2d54f-900d-4ca9-b818-722fce4819f8 None None] MFA Rules not processed for user `241c74785dc14bfcae825157e32ba0a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:17:25.045593 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-72f2d54f-900d-4ca9-b818-722fce4819f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:17:25.219854 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:17:25.232313 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1000/2024] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:17:24 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:17:25.237639 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b13b460-f4bc-4d4e-8836-a23ccc4696a6 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:17:25.354524 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b13b460-f4bc-4d4e-8836-a23ccc4696a6 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:17:25.360339 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81373a0-4361-438a-8667-a1039b583445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:17:25.361062 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81373a0-4361-438a-8667-a1039b583445 admin admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:17:25.361062 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81373a0-4361-438a-8667-a1039b583445 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:17:25.361889 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81373a0-4361-438a-8667-a1039b583445 admin admin] PATH_INFO: `/v3/projects` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:17:25.362828 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81373a0-4361-438a-8667-a1039b583445 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:17:25.365384 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81373a0-4361-438a-8667-a1039b583445 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:17:25.400546 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1025/2025] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:17:25 2022] POST /identity/v3/projects => generated 373 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:17:25.405511 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f2d54f-900d-4ca9-b818-722fce4819f8 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:17:25.419837 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f2d54f-900d-4ca9-b818-722fce4819f8 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:17:25.499445 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe5c1ed-7956-4cbe-b2cb-e4f1de45a43f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:17:25.500760 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe5c1ed-7956-4cbe-b2cb-e4f1de45a43f admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:17:25.501145 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe5c1ed-7956-4cbe-b2cb-e4f1de45a43f admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:17:25.501524 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe5c1ed-7956-4cbe-b2cb-e4f1de45a43f admin admin] PATH_INFO: `/v3/users` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:17:25.502516 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe5c1ed-7956-4cbe-b2cb-e4f1de45a43f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:17:25.504711 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe5c1ed-7956-4cbe-b2cb-e4f1de45a43f admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:17:25.533176 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1001/2026] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:17:25 2022] POST /identity/v3/users => generated 293 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 20:17:25.538136 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81373a0-4361-438a-8667-a1039b583445 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:17:25.557951 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81373a0-4361-438a-8667-a1039b583445 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:17:25.566380 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c430445d-fc32-4b48-b191-638797183c27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:17:25.567139 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c430445d-fc32-4b48-b191-638797183c27 admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:17:25.567797 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c430445d-fc32-4b48-b191-638797183c27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:17:25.568193 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c430445d-fc32-4b48-b191-638797183c27 admin admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:17:25.569042 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c430445d-fc32-4b48-b191-638797183c27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:17:25.571008 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c430445d-fc32-4b48-b191-638797183c27 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:17:25.587822 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1026/2027] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:17:25 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:17:25.592470 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe5c1ed-7956-4cbe-b2cb-e4f1de45a43f admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:17:25.614807 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe5c1ed-7956-4cbe-b2cb-e4f1de45a43f admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:17:25.620696 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614c2fa4-8bf9-421f-9bfe-93eedce7421f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:17:25.621909 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614c2fa4-8bf9-421f-9bfe-93eedce7421f admin admin] REQUEST_METHOD: `PUT` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:17:25.622250 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614c2fa4-8bf9-421f-9bfe-93eedce7421f admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:17:25.622573 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614c2fa4-8bf9-421f-9bfe-93eedce7421f admin admin] PATH_INFO: `/v3/projects/228dc52c4c66481fa3140738fc4d3f9b/users/e2e7312949864c1c8ec48a4583c8f756/roles/e0ab41d8ae22474688b5361bcc75e0f0` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:17:25.641777 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614c2fa4-8bf9-421f-9bfe-93eedce7421f admin admin] RBAC: Authorizing `identity:create_grant(project_id=228dc52c4c66481fa3140738fc4d3f9b, user_id=e2e7312949864c1c8ec48a4583c8f756, role_id=e0ab41d8ae22474688b5361bcc75e0f0)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:17:25.648842 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614c2fa4-8bf9-421f-9bfe-93eedce7421f admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:17:25.669883 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1002/2028] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:17:25 2022] PUT /identity/v3/projects/228dc52c4c66481fa3140738fc4d3f9b/users/e2e7312949864c1c8ec48a4583c8f756/roles/e0ab41d8ae22474688b5361bcc75e0f0 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:17:25.674474 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c430445d-fc32-4b48-b191-638797183c27 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:17:25.688447 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c430445d-fc32-4b48-b191-638797183c27 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:17:25.770866 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d31b3fb-37ca-4b2c-ab4e-e294b1298377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:17:25.771651 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d31b3fb-37ca-4b2c-ab4e-e294b1298377 admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:17:25.771940 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d31b3fb-37ca-4b2c-ab4e-e294b1298377 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:17:25.772219 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d31b3fb-37ca-4b2c-ab4e-e294b1298377 admin admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:17:25.772898 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d31b3fb-37ca-4b2c-ab4e-e294b1298377 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:17:25.774893 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d31b3fb-37ca-4b2c-ab4e-e294b1298377 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:17:25.786835 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1027/2029] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:17:25 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:17:25.792398 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614c2fa4-8bf9-421f-9bfe-93eedce7421f admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:17:25.805444 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614c2fa4-8bf9-421f-9bfe-93eedce7421f admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:17:25.811161 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c076e2fd-5c4b-4668-a1b4-41ab57e40b8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:17:25.812018 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c076e2fd-5c4b-4668-a1b4-41ab57e40b8f admin admin] REQUEST_METHOD: `PUT` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:17:25.812377 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c076e2fd-5c4b-4668-a1b4-41ab57e40b8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:17:25.812955 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c076e2fd-5c4b-4668-a1b4-41ab57e40b8f admin admin] PATH_INFO: `/v3/projects/228dc52c4c66481fa3140738fc4d3f9b/users/e2e7312949864c1c8ec48a4583c8f756/roles/e0ab41d8ae22474688b5361bcc75e0f0` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:17:25.815499 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c076e2fd-5c4b-4668-a1b4-41ab57e40b8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=228dc52c4c66481fa3140738fc4d3f9b, user_id=e2e7312949864c1c8ec48a4583c8f756, role_id=e0ab41d8ae22474688b5361bcc75e0f0)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:17:25.819602 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c076e2fd-5c4b-4668-a1b4-41ab57e40b8f admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:17:25.835000 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1003/2030] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:17:25 2022] PUT /identity/v3/projects/228dc52c4c66481fa3140738fc4d3f9b/users/e2e7312949864c1c8ec48a4583c8f756/roles/e0ab41d8ae22474688b5361bcc75e0f0 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:17:25.852117 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93e20ae-baed-4756-a065-859505f6e8a8 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:17:25.852117 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93e20ae-baed-4756-a065-859505f6e8a8 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:17:25.852117 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93e20ae-baed-4756-a065-859505f6e8a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:17:25.894143 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-a93e20ae-baed-4756-a065-859505f6e8a8 None None] MFA Rules not processed for user `e2e7312949864c1c8ec48a4583c8f756`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:17:25.895639 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-a93e20ae-baed-4756-a065-859505f6e8a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:17:25.920433 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:17:26.009494 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1028/2031] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:17:25 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:17:26.022456 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c076e2fd-5c4b-4668-a1b4-41ab57e40b8f admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:17:26.024739 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-c076e2fd-5c4b-4668-a1b4-41ab57e40b8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:17:26.124347 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c076e2fd-5c4b-4668-a1b4-41ab57e40b8f admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:17:26.131834 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f4ae14-4ad4-484b-b111-1754e0afcabb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df1bca993bc142908cc7cf85eabd2b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:17:26.133401 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f4ae14-4ad4-484b-b111-1754e0afcabb service cinder] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:17:26.133828 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f4ae14-4ad4-484b-b111-1754e0afcabb service cinder] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:17:26.134331 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f4ae14-4ad4-484b-b111-1754e0afcabb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:17:26.153557 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f4ae14-4ad4-484b-b111-1754e0afcabb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:17:26.156213 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f4ae14-4ad4-484b-b111-1754e0afcabb service cinder] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:17:26.176371 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1004/2032] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:17:26 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 20:17:26.298309 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93e20ae-baed-4756-a065-859505f6e8a8 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:17:26.321645 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93e20ae-baed-4756-a065-859505f6e8a8 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:17:26.328443 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffe2726-9da8-4543-9845-7d0fc0d3da67 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df1bca993bc142908cc7cf85eabd2b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:17:26.329462 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffe2726-9da8-4543-9845-7d0fc0d3da67 service cinder] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:17:26.330888 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffe2726-9da8-4543-9845-7d0fc0d3da67 service cinder] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:17:26.331386 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffe2726-9da8-4543-9845-7d0fc0d3da67 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:17:26.472974 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffe2726-9da8-4543-9845-7d0fc0d3da67 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:17:26.474614 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffe2726-9da8-4543-9845-7d0fc0d3da67 service cinder] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:17:26.488641 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1029/2033] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:17:26 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:17:31.251190 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f4ae14-4ad4-484b-b111-1754e0afcabb service cinder] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:17:31.252974 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-34f4ae14-4ad4-484b-b111-1754e0afcabb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:17:31.314887 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:17:31.337982 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f4ae14-4ad4-484b-b111-1754e0afcabb service cinder] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:17:31.348534 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906bfdd4-21f4-4f04-b8d5-e1e1126152cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:17:31.349926 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906bfdd4-21f4-4f04-b8d5-e1e1126152cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:17:31.350677 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906bfdd4-21f4-4f04-b8d5-e1e1126152cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:17:31.351194 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906bfdd4-21f4-4f04-b8d5-e1e1126152cf admin admin] PATH_INFO: `/v3/users/9697e8514c1c4676ad67bcab474edd74` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:17:31.354423 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906bfdd4-21f4-4f04-b8d5-e1e1126152cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=9697e8514c1c4676ad67bcab474edd74)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:17:31.358363 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906bfdd4-21f4-4f04-b8d5-e1e1126152cf admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:17:31.419553 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-906bfdd4-21f4-4f04-b8d5-e1e1126152cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9697e8514c1c4676ad67bcab474edd74'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:17:31.428111 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-906bfdd4-21f4-4f04-b8d5-e1e1126152cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9697e8514c1c4676ad67bcab474edd74'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:17:31.428821 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-906bfdd4-21f4-4f04-b8d5-e1e1126152cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9697e8514c1c4676ad67bcab474edd74'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:17:31.429454 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-906bfdd4-21f4-4f04-b8d5-e1e1126152cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9697e8514c1c4676ad67bcab474edd74'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:17:31.438520 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-906bfdd4-21f4-4f04-b8d5-e1e1126152cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9697e8514c1c4676ad67bcab474edd74'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:17:31.457727 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1005/2034] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:17:31 2022] DELETE /identity/v3/users/9697e8514c1c4676ad67bcab474edd74 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:17:31.462571 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63805ffb-7425-4985-b738-457e307c68e4 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:17:31.463139 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63805ffb-7425-4985-b738-457e307c68e4 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:17:31.463139 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63805ffb-7425-4985-b738-457e307c68e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:17:31.484282 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-63805ffb-7425-4985-b738-457e307c68e4 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:17:31.485642 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-63805ffb-7425-4985-b738-457e307c68e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:17:31.554736 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1030/2035] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:17:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:17:31.565914 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906bfdd4-21f4-4f04-b8d5-e1e1126152cf admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:17:31.567585 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-906bfdd4-21f4-4f04-b8d5-e1e1126152cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:17:31.658742 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906bfdd4-21f4-4f04-b8d5-e1e1126152cf admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:17:31.664298 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a49704-a231-4735-a093-0ff071909ba8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:17:31.665329 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a49704-a231-4735-a093-0ff071909ba8 service neutron] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:17:31.665714 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a49704-a231-4735-a093-0ff071909ba8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:17:31.665993 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a49704-a231-4735-a093-0ff071909ba8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:17:31.678867 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a49704-a231-4735-a093-0ff071909ba8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:17:31.681047 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a49704-a231-4735-a093-0ff071909ba8 service neutron] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:17:31.701232 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1006/2036] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:17:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:17:32.048268 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63805ffb-7425-4985-b738-457e307c68e4 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:17:32.049936 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-63805ffb-7425-4985-b738-457e307c68e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:17:32.063162 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63805ffb-7425-4985-b738-457e307c68e4 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:17:32.072398 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d87c0e-3e7e-4c28-8846-63655ad26749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:17:32.073675 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d87c0e-3e7e-4c28-8846-63655ad26749 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:17:32.074179 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d87c0e-3e7e-4c28-8846-63655ad26749 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:17:32.074680 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d87c0e-3e7e-4c28-8846-63655ad26749 admin admin] PATH_INFO: `/v3/projects/3883c8302985440897fbc397161f297d` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:17:32.076918 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d87c0e-3e7e-4c28-8846-63655ad26749 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3883c8302985440897fbc397161f297d)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:17:32.080748 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d87c0e-3e7e-4c28-8846-63655ad26749 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:17:32.153987 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:17:32.163527 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-c5d87c0e-3e7e-4c28-8846-63655ad26749 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3883c8302985440897fbc397161f297d', 'request_id': 'req-c5d87c0e-3e7e-4c28-8846-63655ad26749'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:17:32.174968 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-c5d87c0e-3e7e-4c28-8846-63655ad26749 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3883c8302985440897fbc397161f297d', 'request_id': 'req-c5d87c0e-3e7e-4c28-8846-63655ad26749'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:17:32.184283 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-c5d87c0e-3e7e-4c28-8846-63655ad26749 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3883c8302985440897fbc397161f297d', 'request_id': 'req-c5d87c0e-3e7e-4c28-8846-63655ad26749'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:17:32.198023 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-c5d87c0e-3e7e-4c28-8846-63655ad26749 admin admin] The token cache is being invalidate because project 3883c8302985440897fbc397161f297d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81772) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:17:32.198470 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-c5d87c0e-3e7e-4c28-8846-63655ad26749 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:17:32.199347 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-c5d87c0e-3e7e-4c28-8846-63655ad26749 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:17:32.201300 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1031/2037] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:17:32 2022] DELETE /identity/v3/projects/3883c8302985440897fbc397161f297d => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:17:32.207046 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da150db-b5f0-4ea2-9ab6-08d92f0cc418 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:17:32.207386 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da150db-b5f0-4ea2-9ab6-08d92f0cc418 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:17:32.207710 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da150db-b5f0-4ea2-9ab6-08d92f0cc418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:17:32.232647 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-6da150db-b5f0-4ea2-9ab6-08d92f0cc418 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:17:32.234213 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-6da150db-b5f0-4ea2-9ab6-08d92f0cc418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:17:32.341960 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1007/2038] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:17:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:17:32.351829 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058d336e-250d-484f-9fd2-9b32257168b7 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:17:32.352142 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058d336e-250d-484f-9fd2-9b32257168b7 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:17:32.352421 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058d336e-250d-484f-9fd2-9b32257168b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:17:32.378066 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-058d336e-250d-484f-9fd2-9b32257168b7 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:17:32.379287 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-058d336e-250d-484f-9fd2-9b32257168b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:17:32.389011 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1032/2039] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:17:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:17:32.393668 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da150db-b5f0-4ea2-9ab6-08d92f0cc418 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:17:32.408679 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da150db-b5f0-4ea2-9ab6-08d92f0cc418 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:17:32.415693 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239fda65-c797-4d28-9181-e402d7ba6892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:17:32.416565 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239fda65-c797-4d28-9181-e402d7ba6892 admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:17:32.416840 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239fda65-c797-4d28-9181-e402d7ba6892 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:17:32.417110 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239fda65-c797-4d28-9181-e402d7ba6892 admin admin] PATH_INFO: `/v3/domains` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:17:32.418025 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239fda65-c797-4d28-9181-e402d7ba6892 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:17:32.418478 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239fda65-c797-4d28-9181-e402d7ba6892 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:17:32.421170 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239fda65-c797-4d28-9181-e402d7ba6892 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:17:32.440907 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1008/2040] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 6 20:17:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:17:32.445506 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058d336e-250d-484f-9fd2-9b32257168b7 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:17:32.457829 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058d336e-250d-484f-9fd2-9b32257168b7 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:17:32.463594 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69aef41-bbe0-4b01-a1d4-b1bbd176d413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:17:32.464305 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69aef41-bbe0-4b01-a1d4-b1bbd176d413 admin admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:17:32.464500 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69aef41-bbe0-4b01-a1d4-b1bbd176d413 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:17:32.464694 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69aef41-bbe0-4b01-a1d4-b1bbd176d413 admin admin] PATH_INFO: `/v3/projects` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:17:32.465359 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69aef41-bbe0-4b01-a1d4-b1bbd176d413 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:17:32.467380 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69aef41-bbe0-4b01-a1d4-b1bbd176d413 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:17:32.510524 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1033/2041] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:17:32 2022] POST /identity/v3/projects => generated 373 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 20:17:32.517729 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239fda65-c797-4d28-9181-e402d7ba6892 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:17:32.527956 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:17:32.540314 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239fda65-c797-4d28-9181-e402d7ba6892 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:17:32.624326 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f3a893-b0c5-4092-8297-bf13a9fa5ae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:17:32.625282 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f3a893-b0c5-4092-8297-bf13a9fa5ae3 admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:17:32.625850 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f3a893-b0c5-4092-8297-bf13a9fa5ae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:17:32.626340 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f3a893-b0c5-4092-8297-bf13a9fa5ae3 admin admin] PATH_INFO: `/v3/users` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:17:32.627639 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f3a893-b0c5-4092-8297-bf13a9fa5ae3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:17:32.631035 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f3a893-b0c5-4092-8297-bf13a9fa5ae3 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:17:32.668084 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1009/2042] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:17:32 2022] POST /identity/v3/users => generated 287 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:17:32.675353 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69aef41-bbe0-4b01-a1d4-b1bbd176d413 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:17:32.688646 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69aef41-bbe0-4b01-a1d4-b1bbd176d413 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:17:32.694436 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed0d1fa-1e4b-488d-9796-124038e20b6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:17:32.695116 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed0d1fa-1e4b-488d-9796-124038e20b6e admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:17:32.695416 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed0d1fa-1e4b-488d-9796-124038e20b6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:17:32.695725 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed0d1fa-1e4b-488d-9796-124038e20b6e admin admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:17:32.696513 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed0d1fa-1e4b-488d-9796-124038e20b6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:17:32.698717 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed0d1fa-1e4b-488d-9796-124038e20b6e admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:17:32.718275 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1034/2043] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:17:32 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:17:32.722717 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f3a893-b0c5-4092-8297-bf13a9fa5ae3 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:17:32.735160 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f3a893-b0c5-4092-8297-bf13a9fa5ae3 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:17:32.740683 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009d4bdb-327e-4aa7-8a9b-58a9a5cd6f30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:17:32.741327 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009d4bdb-327e-4aa7-8a9b-58a9a5cd6f30 admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:17:32.741654 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009d4bdb-327e-4aa7-8a9b-58a9a5cd6f30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:17:32.741939 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009d4bdb-327e-4aa7-8a9b-58a9a5cd6f30 admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:17:32.742634 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009d4bdb-327e-4aa7-8a9b-58a9a5cd6f30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:17:32.744430 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009d4bdb-327e-4aa7-8a9b-58a9a5cd6f30 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:17:32.757710 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1010/2044] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:17:32 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:17:32.762355 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed0d1fa-1e4b-488d-9796-124038e20b6e admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:17:32.774505 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed0d1fa-1e4b-488d-9796-124038e20b6e admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:17:32.779727 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfba9e4c-1024-4bde-b9de-18275a3f214b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:17:32.780421 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfba9e4c-1024-4bde-b9de-18275a3f214b admin admin] REQUEST_METHOD: `PUT` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:17:32.780707 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfba9e4c-1024-4bde-b9de-18275a3f214b admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:17:32.780987 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfba9e4c-1024-4bde-b9de-18275a3f214b admin admin] PATH_INFO: `/v3/projects/f98d9167ed114206a35fca262ebd9477/users/022ac24a0c284315a84ad2c60fb16f3a/roles/4eea39e24b06431aa357053fe7907a3d` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:17:32.803123 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfba9e4c-1024-4bde-b9de-18275a3f214b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f98d9167ed114206a35fca262ebd9477, user_id=022ac24a0c284315a84ad2c60fb16f3a, role_id=4eea39e24b06431aa357053fe7907a3d)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:17:32.807572 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfba9e4c-1024-4bde-b9de-18275a3f214b admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:17:32.826590 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1035/2045] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:17:32 2022] PUT /identity/v3/projects/f98d9167ed114206a35fca262ebd9477/users/022ac24a0c284315a84ad2c60fb16f3a/roles/4eea39e24b06431aa357053fe7907a3d => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 20:17:32.846277 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c253494e-404b-4ac2-8577-275cbe0bb22c None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:17:32.846744 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c253494e-404b-4ac2-8577-275cbe0bb22c None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:17:32.847199 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c253494e-404b-4ac2-8577-275cbe0bb22c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:17:32.894896 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-c253494e-404b-4ac2-8577-275cbe0bb22c None None] MFA Rules not processed for user `022ac24a0c284315a84ad2c60fb16f3a`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:17:32.896812 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-c253494e-404b-4ac2-8577-275cbe0bb22c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:17:33.022299 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1011/2046] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:17:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:17:33.037864 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfba9e4c-1024-4bde-b9de-18275a3f214b admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:17:33.039994 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-dfba9e4c-1024-4bde-b9de-18275a3f214b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:17:33.121922 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfba9e4c-1024-4bde-b9de-18275a3f214b admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:17:33.127008 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1081015-7cb7-40cd-9caa-5c1bd97bf00f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df1bca993bc142908cc7cf85eabd2b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:17:33.127656 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1081015-7cb7-40cd-9caa-5c1bd97bf00f service cinder] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:17:33.127999 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1081015-7cb7-40cd-9caa-5c1bd97bf00f service cinder] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:17:33.128279 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1081015-7cb7-40cd-9caa-5c1bd97bf00f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:17:33.140629 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1081015-7cb7-40cd-9caa-5c1bd97bf00f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:17:33.142185 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1081015-7cb7-40cd-9caa-5c1bd97bf00f service cinder] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:17:33.152937 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1036/2047] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:17:33 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:17:39.921721 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d492969f-9688-46f9-8644-fefc3f6f713f None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:17:39.923511 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d492969f-9688-46f9-8644-fefc3f6f713f None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:17:39.923843 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d492969f-9688-46f9-8644-fefc3f6f713f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:17:39.948811 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-d492969f-9688-46f9-8644-fefc3f6f713f None None] MFA Rules not processed for user `1a52117df70447dba9bbee35d083e44f`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:17:39.950834 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-d492969f-9688-46f9-8644-fefc3f6f713f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:17:40.070014 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1012/2048] 10.222.0.20 () {58 vars in 1009 bytes} [Tue Sep 6 20:17:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:17:40.141286 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1081015-7cb7-40cd-9caa-5c1bd97bf00f service cinder] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:17:40.145492 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-c1081015-7cb7-40cd-9caa-5c1bd97bf00f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:17:40.281080 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1081015-7cb7-40cd-9caa-5c1bd97bf00f service cinder] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:17:40.284894 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a415e38-9043-426a-86cd-67b7a4408e28 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b72516236da47e292d78de201b0552b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:17:40.285519 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a415e38-9043-426a-86cd-67b7a4408e28 service swift] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:17:40.285889 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a415e38-9043-426a-86cd-67b7a4408e28 service swift] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:17:40.286167 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a415e38-9043-426a-86cd-67b7a4408e28 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:17:40.304429 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a415e38-9043-426a-86cd-67b7a4408e28 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:17:40.306378 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a415e38-9043-426a-86cd-67b7a4408e28 service swift] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:17:40.320422 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:17:40.332184 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1037/2049] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Sep 6 20:17:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 20:17:49.447591 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08565b06-eaeb-4d96-b05c-90846d5c1ea4 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:17:49.448117 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08565b06-eaeb-4d96-b05c-90846d5c1ea4 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:17:49.448519 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08565b06-eaeb-4d96-b05c-90846d5c1ea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:17:49.517881 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-08565b06-eaeb-4d96-b05c-90846d5c1ea4 None None] MFA Rules not processed for user `ecde3e4bfd8b4f3490edd3832cea9e48`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:17:49.519515 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-08565b06-eaeb-4d96-b05c-90846d5c1ea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:17:49.552759 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:17:49.705384 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1013/2050] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:17:49 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:17:52.055063 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a415e38-9043-426a-86cd-67b7a4408e28 service swift] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:17:52.421989 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a415e38-9043-426a-86cd-67b7a4408e28 service swift] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:17:52.453260 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac76b799-cd8c-4775-8df0-1bf0e645a8ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:17:52.454010 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac76b799-cd8c-4775-8df0-1bf0e645a8ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:17:52.454218 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac76b799-cd8c-4775-8df0-1bf0e645a8ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:17:52.454526 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac76b799-cd8c-4775-8df0-1bf0e645a8ac admin admin] PATH_INFO: `/v3/users/022ac24a0c284315a84ad2c60fb16f3a` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:17:52.457327 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac76b799-cd8c-4775-8df0-1bf0e645a8ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=022ac24a0c284315a84ad2c60fb16f3a)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:17:52.460151 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac76b799-cd8c-4775-8df0-1bf0e645a8ac admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:17:52.582251 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-ac76b799-cd8c-4775-8df0-1bf0e645a8ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '022ac24a0c284315a84ad2c60fb16f3a'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:17:52.584150 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-ac76b799-cd8c-4775-8df0-1bf0e645a8ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '022ac24a0c284315a84ad2c60fb16f3a'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:17:52.628076 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-ac76b799-cd8c-4775-8df0-1bf0e645a8ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '022ac24a0c284315a84ad2c60fb16f3a'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:17:52.629518 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-ac76b799-cd8c-4775-8df0-1bf0e645a8ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '022ac24a0c284315a84ad2c60fb16f3a'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:17:52.661886 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-ac76b799-cd8c-4775-8df0-1bf0e645a8ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '022ac24a0c284315a84ad2c60fb16f3a'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:17:52.715903 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1038/2051] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:17:52 2022] DELETE /identity/v3/users/022ac24a0c284315a84ad2c60fb16f3a => generated 0 bytes in 662 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:17:52.722604 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae6289c-60ff-4915-b502-cc0225a32ac2 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:17:52.724018 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae6289c-60ff-4915-b502-cc0225a32ac2 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:17:52.724618 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae6289c-60ff-4915-b502-cc0225a32ac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:17:52.808348 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-aae6289c-60ff-4915-b502-cc0225a32ac2 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:17:52.813905 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-aae6289c-60ff-4915-b502-cc0225a32ac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:17:52.964518 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1014/2052] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:17:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:17:52.993533 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac76b799-cd8c-4775-8df0-1bf0e645a8ac admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:17:52.995880 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-ac76b799-cd8c-4775-8df0-1bf0e645a8ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:17:53.287511 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac76b799-cd8c-4775-8df0-1bf0e645a8ac admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:17:53.306039 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283c85b6-04cc-441c-8548-dbdb9a1e78e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:17:53.310400 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283c85b6-04cc-441c-8548-dbdb9a1e78e3 service neutron] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:17:53.310893 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283c85b6-04cc-441c-8548-dbdb9a1e78e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:17:53.311385 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283c85b6-04cc-441c-8548-dbdb9a1e78e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:17:53.340123 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283c85b6-04cc-441c-8548-dbdb9a1e78e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:17:53.343893 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283c85b6-04cc-441c-8548-dbdb9a1e78e3 service neutron] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:17:53.403747 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1039/2053] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:17:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:17:54.077028 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae6289c-60ff-4915-b502-cc0225a32ac2 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:17:54.081511 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-aae6289c-60ff-4915-b502-cc0225a32ac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:17:54.139564 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae6289c-60ff-4915-b502-cc0225a32ac2 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:17:54.147886 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73043b05-860e-40ce-8649-a384650892fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:17:54.149292 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73043b05-860e-40ce-8649-a384650892fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:17:54.149858 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73043b05-860e-40ce-8649-a384650892fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:17:54.150378 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73043b05-860e-40ce-8649-a384650892fa admin admin] PATH_INFO: `/v3/projects/f98d9167ed114206a35fca262ebd9477` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:17:54.153026 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73043b05-860e-40ce-8649-a384650892fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=f98d9167ed114206a35fca262ebd9477)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:17:54.157985 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73043b05-860e-40ce-8649-a384650892fa admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:17:54.278956 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-73043b05-860e-40ce-8649-a384650892fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f98d9167ed114206a35fca262ebd9477', 'request_id': 'req-73043b05-860e-40ce-8649-a384650892fa'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:17:54.288921 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-73043b05-860e-40ce-8649-a384650892fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f98d9167ed114206a35fca262ebd9477', 'request_id': 'req-73043b05-860e-40ce-8649-a384650892fa'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:17:54.329671 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-73043b05-860e-40ce-8649-a384650892fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f98d9167ed114206a35fca262ebd9477', 'request_id': 'req-73043b05-860e-40ce-8649-a384650892fa'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:17:54.341094 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-73043b05-860e-40ce-8649-a384650892fa admin admin] The token cache is being invalidate because project f98d9167ed114206a35fca262ebd9477 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81771) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:17:54.341406 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-73043b05-860e-40ce-8649-a384650892fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:17:54.342239 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-73043b05-860e-40ce-8649-a384650892fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:17:54.344694 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1015/2054] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:17:54 2022] DELETE /identity/v3/projects/f98d9167ed114206a35fca262ebd9477 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:17:54.350232 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9b174f-477c-4710-92bd-6fc0e3925531 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:17:54.350851 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9b174f-477c-4710-92bd-6fc0e3925531 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:17:54.350851 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9b174f-477c-4710-92bd-6fc0e3925531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:17:54.406665 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-0c9b174f-477c-4710-92bd-6fc0e3925531 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:17:54.409139 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-0c9b174f-477c-4710-92bd-6fc0e3925531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:17:54.469495 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:17:54.701877 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1040/2055] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:17:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 352 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:17:54.703848 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c2db7f-ebcc-48c1-8674-7e60aefce2a9 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:17:54.704336 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c2db7f-ebcc-48c1-8674-7e60aefce2a9 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:17:54.705577 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c2db7f-ebcc-48c1-8674-7e60aefce2a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:17:54.830242 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-10c2db7f-ebcc-48c1-8674-7e60aefce2a9 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:17:54.832351 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-10c2db7f-ebcc-48c1-8674-7e60aefce2a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:17:54.853714 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1016/2056] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:17:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:17:54.860315 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9b174f-477c-4710-92bd-6fc0e3925531 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:17:54.894705 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9b174f-477c-4710-92bd-6fc0e3925531 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:17:54.909802 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729121f1-7efb-4216-8814-2096a4445e5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:17:54.911580 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729121f1-7efb-4216-8814-2096a4445e5a admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:17:54.912747 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729121f1-7efb-4216-8814-2096a4445e5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:17:54.915340 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729121f1-7efb-4216-8814-2096a4445e5a admin admin] PATH_INFO: `/v3/domains` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:17:54.919976 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729121f1-7efb-4216-8814-2096a4445e5a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:17:54.923972 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729121f1-7efb-4216-8814-2096a4445e5a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:17:54.929940 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729121f1-7efb-4216-8814-2096a4445e5a admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:17:54.973568 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1041/2057] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 6 20:17:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:17:54.978900 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c2db7f-ebcc-48c1-8674-7e60aefce2a9 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:17:55.004405 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c2db7f-ebcc-48c1-8674-7e60aefce2a9 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:17:55.024196 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8140e88-1464-4934-ae2c-fcb35141ff5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:17:55.025474 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8140e88-1464-4934-ae2c-fcb35141ff5b admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:17:55.027008 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8140e88-1464-4934-ae2c-fcb35141ff5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:17:55.027790 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8140e88-1464-4934-ae2c-fcb35141ff5b admin admin] PATH_INFO: `/v3/projects` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:17:55.029232 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8140e88-1464-4934-ae2c-fcb35141ff5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:17:55.036072 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8140e88-1464-4934-ae2c-fcb35141ff5b admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:17:55.100142 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1017/2058] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:17:54 2022] POST /identity/v3/projects => generated 371 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:17:55.106744 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729121f1-7efb-4216-8814-2096a4445e5a admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:17:55.123668 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729121f1-7efb-4216-8814-2096a4445e5a admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:17:55.341005 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d491d0-672e-4d98-ba4f-fc70f1c31a24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:17:55.342347 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d491d0-672e-4d98-ba4f-fc70f1c31a24 admin admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:17:55.342981 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d491d0-672e-4d98-ba4f-fc70f1c31a24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:17:55.344836 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d491d0-672e-4d98-ba4f-fc70f1c31a24 admin admin] PATH_INFO: `/v3/users` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:17:55.345479 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d491d0-672e-4d98-ba4f-fc70f1c31a24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:17:55.349980 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d491d0-672e-4d98-ba4f-fc70f1c31a24 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:17:55.399795 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1042/2059] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:17:55 2022] POST /identity/v3/users => generated 286 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 20:17:55.401030 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8140e88-1464-4934-ae2c-fcb35141ff5b admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:17:55.421015 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8140e88-1464-4934-ae2c-fcb35141ff5b admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:17:55.444388 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8286f0dc-9898-42e1-9c36-98f5e4368a3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:17:55.445691 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8286f0dc-9898-42e1-9c36-98f5e4368a3d admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:17:55.446190 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8286f0dc-9898-42e1-9c36-98f5e4368a3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:17:55.446663 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8286f0dc-9898-42e1-9c36-98f5e4368a3d admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:17:55.447548 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8286f0dc-9898-42e1-9c36-98f5e4368a3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:17:55.449469 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8286f0dc-9898-42e1-9c36-98f5e4368a3d admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:17:55.482885 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1018/2060] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:17:55 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:17:55.490205 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d491d0-672e-4d98-ba4f-fc70f1c31a24 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:17:55.507110 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d491d0-672e-4d98-ba4f-fc70f1c31a24 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:17:55.523109 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbe8c65-b642-4580-98db-8d26b12c80bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:17:55.526747 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbe8c65-b642-4580-98db-8d26b12c80bb admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:17:55.526747 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbe8c65-b642-4580-98db-8d26b12c80bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:17:55.526747 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbe8c65-b642-4580-98db-8d26b12c80bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:17:55.530465 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbe8c65-b642-4580-98db-8d26b12c80bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:17:55.538402 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbe8c65-b642-4580-98db-8d26b12c80bb admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:17:55.576172 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1043/2061] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:17:55 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:17:55.581262 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8286f0dc-9898-42e1-9c36-98f5e4368a3d admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:17:55.606179 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:17:55.615048 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8286f0dc-9898-42e1-9c36-98f5e4368a3d admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:17:55.625784 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7f4a48-c0a2-44fe-ae8e-c49ede278179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:17:55.629222 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7f4a48-c0a2-44fe-ae8e-c49ede278179 admin admin] REQUEST_METHOD: `PUT` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:17:55.630546 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7f4a48-c0a2-44fe-ae8e-c49ede278179 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:17:55.631539 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7f4a48-c0a2-44fe-ae8e-c49ede278179 admin admin] PATH_INFO: `/v3/projects/b030ea0d617b4583a2397921c884aaa5/users/aaa686e692444eae835ca343926e0595/roles/4eea39e24b06431aa357053fe7907a3d` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:17:55.677374 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7f4a48-c0a2-44fe-ae8e-c49ede278179 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b030ea0d617b4583a2397921c884aaa5, user_id=aaa686e692444eae835ca343926e0595, role_id=4eea39e24b06431aa357053fe7907a3d)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:17:55.682650 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7f4a48-c0a2-44fe-ae8e-c49ede278179 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:17:55.705630 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1019/2062] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:17:55 2022] PUT /identity/v3/projects/b030ea0d617b4583a2397921c884aaa5/users/aaa686e692444eae835ca343926e0595/roles/4eea39e24b06431aa357053fe7907a3d => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:17:55.711615 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8458a240-258f-4d33-ad45-61e8705d288f None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:17:55.714730 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8458a240-258f-4d33-ad45-61e8705d288f None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:17:55.715282 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8458a240-258f-4d33-ad45-61e8705d288f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:17:55.764047 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-8458a240-258f-4d33-ad45-61e8705d288f None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:17:55.766297 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-8458a240-258f-4d33-ad45-61e8705d288f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:17:55.978114 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1044/2063] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:17:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:17:56.004165 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7f4a48-c0a2-44fe-ae8e-c49ede278179 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:17:56.007402 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-7f7f4a48-c0a2-44fe-ae8e-c49ede278179 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:17:56.300854 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc5d4a2-3361-4aa6-acb1-759249f74962 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:17:56.301451 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc5d4a2-3361-4aa6-acb1-759249f74962 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:17:56.302972 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc5d4a2-3361-4aa6-acb1-759249f74962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:17:56.315445 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7f4a48-c0a2-44fe-ae8e-c49ede278179 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:17:56.319906 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dc2462-64a2-47b4-8846-b233ea6b0275 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:17:56.322064 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dc2462-64a2-47b4-8846-b233ea6b0275 service neutron] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:17:56.323763 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dc2462-64a2-47b4-8846-b233ea6b0275 service neutron] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:17:56.323763 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dc2462-64a2-47b4-8846-b233ea6b0275 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:17:56.340695 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dc2462-64a2-47b4-8846-b233ea6b0275 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:17:56.344165 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dc2462-64a2-47b4-8846-b233ea6b0275 service neutron] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:17:56.363431 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-5bc5d4a2-3361-4aa6-acb1-759249f74962 None None] MFA Rules not processed for user `ad400ce73b644938aef19e2da731c4dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:17:56.364820 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1020/2064] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:17:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:17:56.367833 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-5bc5d4a2-3361-4aa6-acb1-759249f74962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:17:56.431489 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:17:56.518788 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1045/2065] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:17:56 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:18:05.593187 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dc2462-64a2-47b4-8846-b233ea6b0275 service neutron] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:18:05.594883 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-d4dc2462-64a2-47b4-8846-b233ea6b0275 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:18:05.791627 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dc2462-64a2-47b4-8846-b233ea6b0275 service neutron] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:18:05.819002 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227d0086-487f-44ae-80e3-4d3d361340b4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f57872de7bb74ea48d30e93e51830aba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:18:05.819620 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227d0086-487f-44ae-80e3-4d3d361340b4 service placement] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:18:05.819943 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227d0086-487f-44ae-80e3-4d3d361340b4 service placement] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:18:05.820223 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227d0086-487f-44ae-80e3-4d3d361340b4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:18:05.829394 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-227d0086-487f-44ae-80e3-4d3d361340b4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:18:05.859457 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227d0086-487f-44ae-80e3-4d3d361340b4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:18:05.861426 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227d0086-487f-44ae-80e3-4d3d361340b4 service placement] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:18:05.894092 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1021/2066] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:18:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:18:06.502762 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=12 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:18:12.540477 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8fec8d-011b-4f56-84e5-ebd61f8061dc None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:18:12.541215 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8fec8d-011b-4f56-84e5-ebd61f8061dc None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:18:12.541215 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8fec8d-011b-4f56-84e5-ebd61f8061dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:18:12.587280 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-2e8fec8d-011b-4f56-84e5-ebd61f8061dc None None] MFA Rules not processed for user `1a52117df70447dba9bbee35d083e44f`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:18:12.588662 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-2e8fec8d-011b-4f56-84e5-ebd61f8061dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:18:12.607500 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d0dd15-6832-404c-a182-51b9974d1982 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:18:12.609383 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d0dd15-6832-404c-a182-51b9974d1982 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:18:12.609905 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d0dd15-6832-404c-a182-51b9974d1982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:18:12.704619 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-51d0dd15-6832-404c-a182-51b9974d1982 None None] MFA Rules not processed for user `aaa686e692444eae835ca343926e0595`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:18:12.709943 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-51d0dd15-6832-404c-a182-51b9974d1982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:18:12.827349 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1046/2067] 10.222.0.20 () {58 vars in 1009 bytes} [Tue Sep 6 20:18:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:18:12.856864 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8fec8d-011b-4f56-84e5-ebd61f8061dc None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:18:12.858806 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-2e8fec8d-011b-4f56-84e5-ebd61f8061dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:18:12.911247 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1022/2068] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:18:12 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:18:12.912442 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d0dd15-6832-404c-a182-51b9974d1982 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:18:12.928801 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:18:12.946551 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d0dd15-6832-404c-a182-51b9974d1982 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:18:12.964814 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6737e0-a8db-4779-850d-74f58469f8c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:18:12.965562 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6737e0-a8db-4779-850d-74f58469f8c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:18:12.965905 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6737e0-a8db-4779-850d-74f58469f8c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:18:12.966190 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6737e0-a8db-4779-850d-74f58469f8c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:18:12.971277 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6737e0-a8db-4779-850d-74f58469f8c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:18:12.981321 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6737e0-a8db-4779-850d-74f58469f8c1 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:18:13.105575 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1023/2069] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:18:12 2022] POST /identity/v3/projects => generated 371 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:18:13.112542 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6737e0-a8db-4779-850d-74f58469f8c1 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:18:13.201806 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8fec8d-011b-4f56-84e5-ebd61f8061dc None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:18:13.382937 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5902dd-fddc-4f00-ad78-bc3a8b3dac9e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b72516236da47e292d78de201b0552b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:18:13.386923 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5902dd-fddc-4f00-ad78-bc3a8b3dac9e service swift] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:18:13.387228 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5902dd-fddc-4f00-ad78-bc3a8b3dac9e service swift] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:18:13.387721 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5902dd-fddc-4f00-ad78-bc3a8b3dac9e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:18:13.498136 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6737e0-a8db-4779-850d-74f58469f8c1 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:18:13.538422 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:18:13.567234 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5902dd-fddc-4f00-ad78-bc3a8b3dac9e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:18:13.568506 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5902dd-fddc-4f00-ad78-bc3a8b3dac9e service swift] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:18:13.587674 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1047/2070] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Sep 6 20:18:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 731 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 06 20:18:13.591400 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b6b468-bf57-4739-bd56-56923a9374a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:18:13.599353 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b6b468-bf57-4739-bd56-56923a9374a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:18:13.607670 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b6b468-bf57-4739-bd56-56923a9374a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:18:13.611888 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b6b468-bf57-4739-bd56-56923a9374a9 admin admin] PATH_INFO: `/v3/users` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:18:13.619220 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b6b468-bf57-4739-bd56-56923a9374a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:18:13.647297 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b6b468-bf57-4739-bd56-56923a9374a9 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:18:13.779477 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1024/2071] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:18:13 2022] POST /identity/v3/users => generated 292 bytes in 673 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:18:13.784275 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5902dd-fddc-4f00-ad78-bc3a8b3dac9e service swift] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:18:13.824450 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5902dd-fddc-4f00-ad78-bc3a8b3dac9e service swift] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:18:13.846184 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f0d625-928f-4b28-aaef-afda2024d2ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:18:13.846819 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f0d625-928f-4b28-aaef-afda2024d2ef admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:18:13.851842 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f0d625-928f-4b28-aaef-afda2024d2ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:18:13.852260 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f0d625-928f-4b28-aaef-afda2024d2ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:18:13.853115 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f0d625-928f-4b28-aaef-afda2024d2ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:18:13.861931 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f0d625-928f-4b28-aaef-afda2024d2ef admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:18:13.909648 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1048/2072] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:18:13 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:18:13.915876 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b6b468-bf57-4739-bd56-56923a9374a9 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:18:13.977333 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b6b468-bf57-4739-bd56-56923a9374a9 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:18:14.013311 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cef2fc7-0968-49d1-8847-365cae7f52ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:18:14.025873 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cef2fc7-0968-49d1-8847-365cae7f52ae admin admin] REQUEST_METHOD: `PUT` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:18:14.025873 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cef2fc7-0968-49d1-8847-365cae7f52ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:18:14.025873 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cef2fc7-0968-49d1-8847-365cae7f52ae admin admin] PATH_INFO: `/v3/projects/d954cb34eed447b793bb6a493dae972a/users/3c8660aeec144ec3af7a272dca472f2f/roles/e0ab41d8ae22474688b5361bcc75e0f0` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:18:14.140608 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cef2fc7-0968-49d1-8847-365cae7f52ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=d954cb34eed447b793bb6a493dae972a, user_id=3c8660aeec144ec3af7a272dca472f2f, role_id=e0ab41d8ae22474688b5361bcc75e0f0)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:18:14.150293 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cef2fc7-0968-49d1-8847-365cae7f52ae admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:18:14.265925 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1025/2073] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:18:13 2022] PUT /identity/v3/projects/d954cb34eed447b793bb6a493dae972a/users/3c8660aeec144ec3af7a272dca472f2f/roles/e0ab41d8ae22474688b5361bcc75e0f0 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:18:14.270928 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f0d625-928f-4b28-aaef-afda2024d2ef admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:18:14.333825 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f0d625-928f-4b28-aaef-afda2024d2ef admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:18:14.651912 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8085bf07-159b-45eb-83b9-45ffccb44641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:18:14.656893 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8085bf07-159b-45eb-83b9-45ffccb44641 admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:18:14.662004 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8085bf07-159b-45eb-83b9-45ffccb44641 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:18:14.664041 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8085bf07-159b-45eb-83b9-45ffccb44641 admin admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:18:14.671007 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8085bf07-159b-45eb-83b9-45ffccb44641 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:18:14.680164 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8085bf07-159b-45eb-83b9-45ffccb44641 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:18:14.746186 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1049/2074] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:18:14 2022] GET /identity/v3/roles => generated 1420 bytes in 471 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:18:14.747431 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cef2fc7-0968-49d1-8847-365cae7f52ae admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:18:14.773706 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cef2fc7-0968-49d1-8847-365cae7f52ae admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:18:14.795925 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ff6788-87e8-4ecc-9c0c-6efa46c9ef70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:18:14.797178 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ff6788-87e8-4ecc-9c0c-6efa46c9ef70 admin admin] REQUEST_METHOD: `PUT` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:18:14.797514 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ff6788-87e8-4ecc-9c0c-6efa46c9ef70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:18:14.797839 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ff6788-87e8-4ecc-9c0c-6efa46c9ef70 admin admin] PATH_INFO: `/v3/projects/d954cb34eed447b793bb6a493dae972a/users/3c8660aeec144ec3af7a272dca472f2f/roles/e0ab41d8ae22474688b5361bcc75e0f0` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:18:14.822120 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ff6788-87e8-4ecc-9c0c-6efa46c9ef70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d954cb34eed447b793bb6a493dae972a, user_id=3c8660aeec144ec3af7a272dca472f2f, role_id=e0ab41d8ae22474688b5361bcc75e0f0)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:18:14.824897 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ff6788-87e8-4ecc-9c0c-6efa46c9ef70 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:18:14.898828 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1026/2075] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:18:14 2022] PUT /identity/v3/projects/d954cb34eed447b793bb6a493dae972a/users/3c8660aeec144ec3af7a272dca472f2f/roles/e0ab41d8ae22474688b5361bcc75e0f0 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 20:18:24.703852 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=39 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:18:24.867437 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=52,keystone:INSERT=6 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:18:31.583606 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8931253-a6a2-4f0b-91b2-900824cba63e None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:18:31.584935 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8931253-a6a2-4f0b-91b2-900824cba63e None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:18:31.585385 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8931253-a6a2-4f0b-91b2-900824cba63e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:18:31.660818 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-a8931253-a6a2-4f0b-91b2-900824cba63e None None] MFA Rules not processed for user `1a52117df70447dba9bbee35d083e44f`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:18:31.663939 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-a8931253-a6a2-4f0b-91b2-900824cba63e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:18:32.040964 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1050/2076] 10.222.0.20 () {58 vars in 967 bytes} [Tue Sep 6 20:18:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 460 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:18:32.071428 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ff6788-87e8-4ecc-9c0c-6efa46c9ef70 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:18:32.435506 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ff6788-87e8-4ecc-9c0c-6efa46c9ef70 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:18:32.466528 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ab5e33-69ca-4e4d-bf96-ce16048fdf02 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b72516236da47e292d78de201b0552b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:18:32.468887 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ab5e33-69ca-4e4d-bf96-ce16048fdf02 service swift] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:18:32.471543 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ab5e33-69ca-4e4d-bf96-ce16048fdf02 service swift] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:18:32.473478 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ab5e33-69ca-4e4d-bf96-ce16048fdf02 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:18:32.518612 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ab5e33-69ca-4e4d-bf96-ce16048fdf02 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:18:32.521667 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ab5e33-69ca-4e4d-bf96-ce16048fdf02 service swift] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:18:32.555626 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1027/2077] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Sep 6 20:18:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 20:18:34.106882 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f7326d-8b41-4198-8d79-731b3f607e87 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:18:34.107329 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f7326d-8b41-4198-8d79-731b3f607e87 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:18:34.107734 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f7326d-8b41-4198-8d79-731b3f607e87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:18:34.335380 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-27f7326d-8b41-4198-8d79-731b3f607e87 None None] MFA Rules not processed for user `3c8660aeec144ec3af7a272dca472f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:18:34.339491 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-27f7326d-8b41-4198-8d79-731b3f607e87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:18:34.666353 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1051/2078] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:18:34 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 562 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:18:34.741318 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ab5e33-69ca-4e4d-bf96-ce16048fdf02 service swift] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:18:34.746759 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-95ab5e33-69ca-4e4d-bf96-ce16048fdf02 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:18:34.932667 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f7326d-8b41-4198-8d79-731b3f607e87 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:18:34.936550 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-27f7326d-8b41-4198-8d79-731b3f607e87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:18:35.019243 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ab5e33-69ca-4e4d-bf96-ce16048fdf02 service swift] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:18:35.033938 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f1baa4-9f9e-47af-974a-e9ccd5f7d2f9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5776cbf4f6714b2083c3ebdd6d98cc87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:18:35.034923 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f1baa4-9f9e-47af-974a-e9ccd5f7d2f9 service nova] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:18:35.035406 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f1baa4-9f9e-47af-974a-e9ccd5f7d2f9 service nova] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:18:35.038237 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f1baa4-9f9e-47af-974a-e9ccd5f7d2f9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:18:35.190709 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f7326d-8b41-4198-8d79-731b3f607e87 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:18:35.204549 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f3b88c-6ee2-401e-b910-f0837adf01aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:18:35.205553 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f3b88c-6ee2-401e-b910-f0837adf01aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:18:35.205985 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f3b88c-6ee2-401e-b910-f0837adf01aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:18:35.206517 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f3b88c-6ee2-401e-b910-f0837adf01aa admin admin] PATH_INFO: `/v3/users/241c74785dc14bfcae825157e32ba0a4` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:18:35.208326 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f3b88c-6ee2-401e-b910-f0837adf01aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=241c74785dc14bfcae825157e32ba0a4)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:18:35.210684 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f3b88c-6ee2-401e-b910-f0837adf01aa admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:18:35.275465 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f1baa4-9f9e-47af-974a-e9ccd5f7d2f9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:18:35.279642 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f1baa4-9f9e-47af-974a-e9ccd5f7d2f9 service nova] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:18:35.309850 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:18:35.311898 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1028/2079] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:18:34 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 571 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:18:35.333652 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-62f3b88c-6ee2-401e-b910-f0837adf01aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '241c74785dc14bfcae825157e32ba0a4'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:18:35.334292 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-62f3b88c-6ee2-401e-b910-f0837adf01aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '241c74785dc14bfcae825157e32ba0a4'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:18:35.365403 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-62f3b88c-6ee2-401e-b910-f0837adf01aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '241c74785dc14bfcae825157e32ba0a4'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:18:35.366775 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-62f3b88c-6ee2-401e-b910-f0837adf01aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '241c74785dc14bfcae825157e32ba0a4'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:18:35.382885 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-62f3b88c-6ee2-401e-b910-f0837adf01aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '241c74785dc14bfcae825157e32ba0a4'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:18:35.424230 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1052/2080] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:18:34 2022] DELETE /identity/v3/users/241c74785dc14bfcae825157e32ba0a4 => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:18:35.429841 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ea3fdf-75ca-46df-91a0-63faa7256e57 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:18:35.430418 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ea3fdf-75ca-46df-91a0-63faa7256e57 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:18:35.430848 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ea3fdf-75ca-46df-91a0-63faa7256e57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:18:35.488985 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-46ea3fdf-75ca-46df-91a0-63faa7256e57 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:18:35.495119 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-46ea3fdf-75ca-46df-91a0-63faa7256e57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:18:35.610960 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:18:35.885490 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1029/2081] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:18:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 456 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:18:35.906229 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f3b88c-6ee2-401e-b910-f0837adf01aa admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:18:35.909026 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-62f3b88c-6ee2-401e-b910-f0837adf01aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:18:36.208443 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f3b88c-6ee2-401e-b910-f0837adf01aa admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:18:36.218798 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6055a55-6ebc-4f94-8e3d-03721706306f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:18:36.220826 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6055a55-6ebc-4f94-8e3d-03721706306f service neutron] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:18:36.221795 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6055a55-6ebc-4f94-8e3d-03721706306f service neutron] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:18:36.222690 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6055a55-6ebc-4f94-8e3d-03721706306f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:18:36.245298 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6055a55-6ebc-4f94-8e3d-03721706306f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:18:36.248678 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6055a55-6ebc-4f94-8e3d-03721706306f service neutron] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:18:36.274648 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1053/2082] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:18:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 20:18:38.021783 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ea3fdf-75ca-46df-91a0-63faa7256e57 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:18:38.025582 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-46ea3fdf-75ca-46df-91a0-63faa7256e57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:18:38.077941 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ea3fdf-75ca-46df-91a0-63faa7256e57 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:18:38.098239 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a84151-dc8b-4801-8660-b615fb17fadd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:18:38.103661 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a84151-dc8b-4801-8660-b615fb17fadd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:18:38.104813 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a84151-dc8b-4801-8660-b615fb17fadd admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:18:38.107200 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a84151-dc8b-4801-8660-b615fb17fadd admin admin] PATH_INFO: `/v3/projects/f87426bf9f944b258c1c80097faeb0c7` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:18:38.113037 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a84151-dc8b-4801-8660-b615fb17fadd admin admin] RBAC: Authorizing `identity:delete_project(project_id=f87426bf9f944b258c1c80097faeb0c7)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:18:38.124216 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a84151-dc8b-4801-8660-b615fb17fadd admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:18:38.462187 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-06a84151-dc8b-4801-8660-b615fb17fadd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f87426bf9f944b258c1c80097faeb0c7', 'request_id': 'req-06a84151-dc8b-4801-8660-b615fb17fadd'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:18:38.486614 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-06a84151-dc8b-4801-8660-b615fb17fadd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f87426bf9f944b258c1c80097faeb0c7', 'request_id': 'req-06a84151-dc8b-4801-8660-b615fb17fadd'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:18:38.510114 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-06a84151-dc8b-4801-8660-b615fb17fadd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f87426bf9f944b258c1c80097faeb0c7', 'request_id': 'req-06a84151-dc8b-4801-8660-b615fb17fadd'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:18:38.539907 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-06a84151-dc8b-4801-8660-b615fb17fadd admin admin] The token cache is being invalidate because project f87426bf9f944b258c1c80097faeb0c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81771) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:18:38.541551 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-06a84151-dc8b-4801-8660-b615fb17fadd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:18:38.544036 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-06a84151-dc8b-4801-8660-b615fb17fadd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:18:38.549867 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1030/2083] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:18:38 2022] DELETE /identity/v3/projects/f87426bf9f944b258c1c80097faeb0c7 => generated 0 bytes in 529 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:18:38.557010 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6055a55-6ebc-4f94-8e3d-03721706306f service neutron] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:18:38.610126 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6055a55-6ebc-4f94-8e3d-03721706306f service neutron] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:18:38.931607 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d32f2d1-0fe4-4f31-b26a-48c12a01b223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:18:38.933293 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d32f2d1-0fe4-4f31-b26a-48c12a01b223 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:18:38.935377 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d32f2d1-0fe4-4f31-b26a-48c12a01b223 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:18:38.936179 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d32f2d1-0fe4-4f31-b26a-48c12a01b223 admin admin] PATH_INFO: `/v3/users/e2e7312949864c1c8ec48a4583c8f756` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:18:38.941944 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d32f2d1-0fe4-4f31-b26a-48c12a01b223 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2e7312949864c1c8ec48a4583c8f756)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:18:38.946271 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d32f2d1-0fe4-4f31-b26a-48c12a01b223 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:18:39.110671 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-7d32f2d1-0fe4-4f31-b26a-48c12a01b223 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2e7312949864c1c8ec48a4583c8f756'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:18:39.112128 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-7d32f2d1-0fe4-4f31-b26a-48c12a01b223 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2e7312949864c1c8ec48a4583c8f756'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:18:39.132223 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:18:39.162421 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-7d32f2d1-0fe4-4f31-b26a-48c12a01b223 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2e7312949864c1c8ec48a4583c8f756'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:18:39.165241 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-7d32f2d1-0fe4-4f31-b26a-48c12a01b223 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2e7312949864c1c8ec48a4583c8f756'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:18:39.184152 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a84151-dc8b-4801-8660-b615fb17fadd admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:18:39.186485 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-06a84151-dc8b-4801-8660-b615fb17fadd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:18:39.199464 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-7d32f2d1-0fe4-4f31-b26a-48c12a01b223 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2e7312949864c1c8ec48a4583c8f756'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:18:39.214990 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a84151-dc8b-4801-8660-b615fb17fadd admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:18:39.222098 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd965e65-5101-425d-b67d-0a30f73a0455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:18:39.223193 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd965e65-5101-425d-b67d-0a30f73a0455 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:18:39.223960 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd965e65-5101-425d-b67d-0a30f73a0455 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:18:39.224568 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd965e65-5101-425d-b67d-0a30f73a0455 admin admin] PATH_INFO: `/v3/users/861d88f0f5674b299575ad2e81dbaaf6` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:18:39.226944 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd965e65-5101-425d-b67d-0a30f73a0455 admin admin] RBAC: Authorizing `identity:delete_user(user_id=861d88f0f5674b299575ad2e81dbaaf6)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:18:39.232016 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd965e65-5101-425d-b67d-0a30f73a0455 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:18:39.261135 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1054/2084] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:18:38 2022] DELETE /identity/v3/users/e2e7312949864c1c8ec48a4583c8f756 => generated 0 bytes in 702 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:18:39.365278 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-dd965e65-5101-425d-b67d-0a30f73a0455 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '861d88f0f5674b299575ad2e81dbaaf6'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:18:39.384126 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-dd965e65-5101-425d-b67d-0a30f73a0455 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '861d88f0f5674b299575ad2e81dbaaf6'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:18:39.385262 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-dd965e65-5101-425d-b67d-0a30f73a0455 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '861d88f0f5674b299575ad2e81dbaaf6'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:18:39.386062 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-dd965e65-5101-425d-b67d-0a30f73a0455 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '861d88f0f5674b299575ad2e81dbaaf6'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:18:39.413130 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-dd965e65-5101-425d-b67d-0a30f73a0455 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '861d88f0f5674b299575ad2e81dbaaf6'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:18:39.544455 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1031/2085] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:18:39 2022] DELETE /identity/v3/users/861d88f0f5674b299575ad2e81dbaaf6 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 20:18:40.683345 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d32f2d1-0fe4-4f31-b26a-48c12a01b223 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:18:40.689400 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-7d32f2d1-0fe4-4f31-b26a-48c12a01b223 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:18:40.917991 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d32f2d1-0fe4-4f31-b26a-48c12a01b223 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:18:41.212615 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4aca66-b711-489f-939f-ef515af00b1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:18:41.215796 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4aca66-b711-489f-939f-ef515af00b1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:18:41.216843 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4aca66-b711-489f-939f-ef515af00b1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:18:41.217391 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4aca66-b711-489f-939f-ef515af00b1c admin admin] PATH_INFO: `/v3/projects/2ca4ba99b8a142d1aaf5ab9df178d981` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:18:41.226259 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4aca66-b711-489f-939f-ef515af00b1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ca4ba99b8a142d1aaf5ab9df178d981)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:18:41.236068 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4aca66-b711-489f-939f-ef515af00b1c admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:18:41.586286 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-2e4aca66-b711-489f-939f-ef515af00b1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ca4ba99b8a142d1aaf5ab9df178d981', 'request_id': 'req-2e4aca66-b711-489f-939f-ef515af00b1c'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:18:41.626993 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-2e4aca66-b711-489f-939f-ef515af00b1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ca4ba99b8a142d1aaf5ab9df178d981', 'request_id': 'req-2e4aca66-b711-489f-939f-ef515af00b1c'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:18:41.689898 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd965e65-5101-425d-b67d-0a30f73a0455 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:18:41.692296 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-dd965e65-5101-425d-b67d-0a30f73a0455 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:18:41.694437 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-2e4aca66-b711-489f-939f-ef515af00b1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ca4ba99b8a142d1aaf5ab9df178d981', 'request_id': 'req-2e4aca66-b711-489f-939f-ef515af00b1c'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:18:41.748124 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-2e4aca66-b711-489f-939f-ef515af00b1c admin admin] The token cache is being invalidate because project 2ca4ba99b8a142d1aaf5ab9df178d981 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81772) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:18:41.748448 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-2e4aca66-b711-489f-939f-ef515af00b1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:18:41.750188 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-2e4aca66-b711-489f-939f-ef515af00b1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:18:41.758578 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1055/2086] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:18:40 2022] DELETE /identity/v3/projects/2ca4ba99b8a142d1aaf5ab9df178d981 => generated 0 bytes in 1076 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:18:41.764099 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a5b942-cfae-48e8-8ad1-b7cec0f0a958 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:18:41.764386 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a5b942-cfae-48e8-8ad1-b7cec0f0a958 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:18:41.766366 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd965e65-5101-425d-b67d-0a30f73a0455 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:18:41.766843 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a5b942-cfae-48e8-8ad1-b7cec0f0a958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:18:41.822779 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-98a5b942-cfae-48e8-8ad1-b7cec0f0a958 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:18:41.825512 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-98a5b942-cfae-48e8-8ad1-b7cec0f0a958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:18:41.839445 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:18:41.887783 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:18:42.007303 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1056/2087] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:18:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:18:42.014103 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6effab5-b483-45e9-9990-bce84669b1c9 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:18:42.014419 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6effab5-b483-45e9-9990-bce84669b1c9 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:18:42.014659 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6effab5-b483-45e9-9990-bce84669b1c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:18:42.063121 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-a6effab5-b483-45e9-9990-bce84669b1c9 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:18:42.064835 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-a6effab5-b483-45e9-9990-bce84669b1c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:18:42.088227 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1057/2088] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:18:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:18:42.098003 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6effab5-b483-45e9-9990-bce84669b1c9 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:18:42.126883 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6effab5-b483-45e9-9990-bce84669b1c9 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:18:42.144552 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d93bbda-060b-4449-b3fb-53afa57f75be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:18:42.146516 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d93bbda-060b-4449-b3fb-53afa57f75be admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:18:42.148522 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d93bbda-060b-4449-b3fb-53afa57f75be admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:18:42.149058 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d93bbda-060b-4449-b3fb-53afa57f75be admin admin] PATH_INFO: `/v3/domains` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:18:42.150038 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d93bbda-060b-4449-b3fb-53afa57f75be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:18:42.151800 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d93bbda-060b-4449-b3fb-53afa57f75be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:18:42.158214 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2e57c4-f932-4982-9a96-3ff0f26c2785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:18:42.158712 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d93bbda-060b-4449-b3fb-53afa57f75be admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:18:42.159399 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2e57c4-f932-4982-9a96-3ff0f26c2785 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:18:42.159801 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2e57c4-f932-4982-9a96-3ff0f26c2785 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:18:42.160183 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2e57c4-f932-4982-9a96-3ff0f26c2785 admin admin] PATH_INFO: `/v3/projects/228dc52c4c66481fa3140738fc4d3f9b` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:18:42.166870 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2e57c4-f932-4982-9a96-3ff0f26c2785 admin admin] RBAC: Authorizing `identity:delete_project(project_id=228dc52c4c66481fa3140738fc4d3f9b)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:18:42.179010 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2e57c4-f932-4982-9a96-3ff0f26c2785 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:18:42.196962 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1058/2089] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 6 20:18:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:18:42.200564 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d93bbda-060b-4449-b3fb-53afa57f75be admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:18:42.225251 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d93bbda-060b-4449-b3fb-53afa57f75be admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:18:42.245849 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2805c2b9-6e01-4e56-ad71-49c4a6c748b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:18:42.245849 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2805c2b9-6e01-4e56-ad71-49c4a6c748b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:18:42.245849 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2805c2b9-6e01-4e56-ad71-49c4a6c748b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:18:42.245849 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2805c2b9-6e01-4e56-ad71-49c4a6c748b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:18:42.246626 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2805c2b9-6e01-4e56-ad71-49c4a6c748b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:18:42.254198 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2805c2b9-6e01-4e56-ad71-49c4a6c748b4 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:18:42.348654 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1059/2090] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:18:42 2022] POST /identity/v3/projects => generated 369 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:18:42.354293 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2805c2b9-6e01-4e56-ad71-49c4a6c748b4 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:18:42.379765 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-0e2e57c4-f932-4982-9a96-3ff0f26c2785 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '228dc52c4c66481fa3140738fc4d3f9b', 'request_id': 'req-0e2e57c4-f932-4982-9a96-3ff0f26c2785'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:18:42.406671 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-0e2e57c4-f932-4982-9a96-3ff0f26c2785 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '228dc52c4c66481fa3140738fc4d3f9b', 'request_id': 'req-0e2e57c4-f932-4982-9a96-3ff0f26c2785'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:18:42.428941 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-0e2e57c4-f932-4982-9a96-3ff0f26c2785 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '228dc52c4c66481fa3140738fc4d3f9b', 'request_id': 'req-0e2e57c4-f932-4982-9a96-3ff0f26c2785'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:18:42.464383 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-0e2e57c4-f932-4982-9a96-3ff0f26c2785 admin admin] The token cache is being invalidate because project 228dc52c4c66481fa3140738fc4d3f9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81771) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:18:42.464909 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-0e2e57c4-f932-4982-9a96-3ff0f26c2785 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:18:42.466067 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-0e2e57c4-f932-4982-9a96-3ff0f26c2785 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:18:42.469597 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1032/2091] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:18:41 2022] DELETE /identity/v3/projects/228dc52c4c66481fa3140738fc4d3f9b => generated 0 bytes in 781 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:18:42.479369 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718415c1-4809-4702-bca4-d57261506342 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:18:42.479876 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718415c1-4809-4702-bca4-d57261506342 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:18:42.480312 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718415c1-4809-4702-bca4-d57261506342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:18:42.577491 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-718415c1-4809-4702-bca4-d57261506342 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:18:42.581663 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-718415c1-4809-4702-bca4-d57261506342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:18:42.656868 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2805c2b9-6e01-4e56-ad71-49c4a6c748b4 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:18:42.672680 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be62e6b-cb81-487c-9de4-89b233f90106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:18:42.673769 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be62e6b-cb81-487c-9de4-89b233f90106 admin admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:18:42.674248 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be62e6b-cb81-487c-9de4-89b233f90106 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:18:42.674725 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be62e6b-cb81-487c-9de4-89b233f90106 admin admin] PATH_INFO: `/v3/users` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:18:42.675879 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be62e6b-cb81-487c-9de4-89b233f90106 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:18:42.688198 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be62e6b-cb81-487c-9de4-89b233f90106 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:18:42.762110 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1060/2092] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:18:42 2022] POST /identity/v3/users => generated 285 bytes in 409 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:18:42.768247 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be62e6b-cb81-487c-9de4-89b233f90106 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:18:42.775794 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-8be62e6b-cb81-487c-9de4-89b233f90106 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:18:42.790756 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1033/2093] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:18:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:18:42.810511 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0660dec-8ef3-457d-87d7-143e0bcf8f73 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:18:42.810909 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0660dec-8ef3-457d-87d7-143e0bcf8f73 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:18:42.811193 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0660dec-8ef3-457d-87d7-143e0bcf8f73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:18:42.823313 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be62e6b-cb81-487c-9de4-89b233f90106 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:18:42.849698 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953efc37-faf4-47d0-a1ca-dc6c6fb36579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:18:42.851089 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953efc37-faf4-47d0-a1ca-dc6c6fb36579 admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:18:42.851818 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953efc37-faf4-47d0-a1ca-dc6c6fb36579 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:18:42.852513 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953efc37-faf4-47d0-a1ca-dc6c6fb36579 admin admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:18:42.853945 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953efc37-faf4-47d0-a1ca-dc6c6fb36579 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:18:42.857433 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953efc37-faf4-47d0-a1ca-dc6c6fb36579 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:18:42.897857 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1061/2094] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:18:42 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 20:18:42.903901 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953efc37-faf4-47d0-a1ca-dc6c6fb36579 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:18:42.920296 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-b0660dec-8ef3-457d-87d7-143e0bcf8f73 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:18:42.930142 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-b0660dec-8ef3-457d-87d7-143e0bcf8f73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:18:42.936447 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953efc37-faf4-47d0-a1ca-dc6c6fb36579 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:18:42.960231 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855a0781-422c-49aa-8dc4-931492f5a3eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:18:42.961247 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855a0781-422c-49aa-8dc4-931492f5a3eb admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:18:42.961771 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855a0781-422c-49aa-8dc4-931492f5a3eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:18:42.962244 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855a0781-422c-49aa-8dc4-931492f5a3eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:18:42.963638 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855a0781-422c-49aa-8dc4-931492f5a3eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:18:42.967007 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855a0781-422c-49aa-8dc4-931492f5a3eb admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:18:43.003582 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1034/2095] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:18:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:18:43.006755 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0660dec-8ef3-457d-87d7-143e0bcf8f73 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:18:43.013466 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1062/2096] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:18:42 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 20:18:43.023306 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855a0781-422c-49aa-8dc4-931492f5a3eb admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:18:43.039839 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0660dec-8ef3-457d-87d7-143e0bcf8f73 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:18:43.050015 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a3a94b-4246-45bf-af2a-663bd725af4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:18:43.051325 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a3a94b-4246-45bf-af2a-663bd725af4b admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:18:43.052133 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a3a94b-4246-45bf-af2a-663bd725af4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:18:43.052724 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a3a94b-4246-45bf-af2a-663bd725af4b admin admin] PATH_INFO: `/v3/domains` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:18:43.053736 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a3a94b-4246-45bf-af2a-663bd725af4b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:18:43.054283 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a3a94b-4246-45bf-af2a-663bd725af4b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:18:43.056963 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a3a94b-4246-45bf-af2a-663bd725af4b admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:18:43.090662 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855a0781-422c-49aa-8dc4-931492f5a3eb admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:18:43.092466 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1035/2097] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 6 20:18:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:18:43.097296 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a3a94b-4246-45bf-af2a-663bd725af4b admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:18:43.127636 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a3a94b-4246-45bf-af2a-663bd725af4b admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:18:43.134283 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40abe635-686d-4204-99c1-e1837592b7de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:18:43.135749 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40abe635-686d-4204-99c1-e1837592b7de admin admin] REQUEST_METHOD: `PUT` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:18:43.136294 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40abe635-686d-4204-99c1-e1837592b7de admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:18:43.136873 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40abe635-686d-4204-99c1-e1837592b7de admin admin] PATH_INFO: `/v3/projects/258bdd28ff27449e8fa3bab0678b6aff/users/060a287ebe6944af8f2947e6af97703c/roles/4eea39e24b06431aa357053fe7907a3d` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:18:43.141764 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d487dbaf-16e7-4b74-9313-9a499cff807d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:18:43.142731 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d487dbaf-16e7-4b74-9313-9a499cff807d admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:18:43.143332 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d487dbaf-16e7-4b74-9313-9a499cff807d admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:18:43.143765 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d487dbaf-16e7-4b74-9313-9a499cff807d admin admin] PATH_INFO: `/v3/projects` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:18:43.144726 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d487dbaf-16e7-4b74-9313-9a499cff807d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:18:43.147055 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d487dbaf-16e7-4b74-9313-9a499cff807d admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:18:43.233569 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40abe635-686d-4204-99c1-e1837592b7de admin admin] RBAC: Authorizing `identity:create_grant(project_id=258bdd28ff27449e8fa3bab0678b6aff, user_id=060a287ebe6944af8f2947e6af97703c, role_id=4eea39e24b06431aa357053fe7907a3d)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:18:43.252713 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1036/2098] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:18:43 2022] POST /identity/v3/projects => generated 369 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:18:43.253967 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40abe635-686d-4204-99c1-e1837592b7de admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:18:43.259368 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d487dbaf-16e7-4b74-9313-9a499cff807d admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:18:43.314087 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1063/2099] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:18:43 2022] PUT /identity/v3/projects/258bdd28ff27449e8fa3bab0678b6aff/users/060a287ebe6944af8f2947e6af97703c/roles/4eea39e24b06431aa357053fe7907a3d => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:18:43.318333 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:18:43.336250 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df8e2ae-f4ff-4415-b229-f39ed4dce763 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:18:43.336769 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df8e2ae-f4ff-4415-b229-f39ed4dce763 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:18:43.337223 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df8e2ae-f4ff-4415-b229-f39ed4dce763 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:18:43.415906 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:18:43.449793 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-8df8e2ae-f4ff-4415-b229-f39ed4dce763 None None] MFA Rules not processed for user `060a287ebe6944af8f2947e6af97703c`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:18:43.454636 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-8df8e2ae-f4ff-4415-b229-f39ed4dce763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:18:43.455196 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d487dbaf-16e7-4b74-9313-9a499cff807d admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:18:43.627139 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50e939b-8f78-4fd3-9d2c-333aca1987ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:18:43.628693 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50e939b-8f78-4fd3-9d2c-333aca1987ed admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:18:43.629160 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50e939b-8f78-4fd3-9d2c-333aca1987ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:18:43.630497 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50e939b-8f78-4fd3-9d2c-333aca1987ed admin admin] PATH_INFO: `/v3/users` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:18:43.631441 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50e939b-8f78-4fd3-9d2c-333aca1987ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:18:43.640784 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50e939b-8f78-4fd3-9d2c-333aca1987ed admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:18:43.661919 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1064/2100] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:18:43 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 322 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:18:43.684260 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df8e2ae-f4ff-4415-b229-f39ed4dce763 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:18:43.689353 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-8df8e2ae-f4ff-4415-b229-f39ed4dce763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:18:43.692556 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1037/2101] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:18:43 2022] POST /identity/v3/users => generated 285 bytes in 435 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:18:43.694805 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50e939b-8f78-4fd3-9d2c-333aca1987ed admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:18:43.775865 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50e939b-8f78-4fd3-9d2c-333aca1987ed admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:18:43.816284 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ddef7c-4fe8-4387-aeaf-91483b9d372f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:18:43.822342 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ddef7c-4fe8-4387-aeaf-91483b9d372f admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:18:43.823753 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ddef7c-4fe8-4387-aeaf-91483b9d372f admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:18:43.824483 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ddef7c-4fe8-4387-aeaf-91483b9d372f admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:18:43.827456 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ddef7c-4fe8-4387-aeaf-91483b9d372f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:18:43.838801 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ddef7c-4fe8-4387-aeaf-91483b9d372f admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:18:43.890781 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1038/2102] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:18:43 2022] GET /identity/v3/roles => generated 1420 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:18:43.893232 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ddef7c-4fe8-4387-aeaf-91483b9d372f admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:18:43.914704 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ddef7c-4fe8-4387-aeaf-91483b9d372f admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:18:43.924730 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3453a5-217b-441f-8fe6-a4f935e4071c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:18:43.925650 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df8e2ae-f4ff-4415-b229-f39ed4dce763 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:18:43.927367 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3453a5-217b-441f-8fe6-a4f935e4071c admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:18:43.928055 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3453a5-217b-441f-8fe6-a4f935e4071c admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:18:43.928741 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3453a5-217b-441f-8fe6-a4f935e4071c admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:18:43.930124 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3453a5-217b-441f-8fe6-a4f935e4071c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:18:43.933993 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3453a5-217b-441f-8fe6-a4f935e4071c admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:18:43.961870 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0118a343-cfa4-4c00-a9e5-a57fb910cbe6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07af18f35ffc42a295278d5349e06381', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:18:43.961870 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0118a343-cfa4-4c00-a9e5-a57fb910cbe6 service glance] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:18:43.961870 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0118a343-cfa4-4c00-a9e5-a57fb910cbe6 service glance] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:18:43.961870 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0118a343-cfa4-4c00-a9e5-a57fb910cbe6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:18:43.963575 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1039/2103] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:18:43 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:18:43.965890 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3453a5-217b-441f-8fe6-a4f935e4071c admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:18:43.986616 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3453a5-217b-441f-8fe6-a4f935e4071c admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:18:43.998766 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7332bda2-599e-4027-806d-aae0dea94411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:18:44.000301 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7332bda2-599e-4027-806d-aae0dea94411 admin admin] REQUEST_METHOD: `PUT` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:18:44.000652 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0118a343-cfa4-4c00-a9e5-a57fb910cbe6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:18:44.001329 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7332bda2-599e-4027-806d-aae0dea94411 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:18:44.002002 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7332bda2-599e-4027-806d-aae0dea94411 admin admin] PATH_INFO: `/v3/projects/1cdc0123b9f34602ab3aeac846b099f6/users/b149cd4e47bc4e109505b5422856bdb9/roles/4eea39e24b06431aa357053fe7907a3d` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:18:44.003113 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0118a343-cfa4-4c00-a9e5-a57fb910cbe6 service glance] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:18:44.037634 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7332bda2-599e-4027-806d-aae0dea94411 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cdc0123b9f34602ab3aeac846b099f6, user_id=b149cd4e47bc4e109505b5422856bdb9, role_id=4eea39e24b06431aa357053fe7907a3d)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:18:44.042756 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7332bda2-599e-4027-806d-aae0dea94411 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:18:44.043573 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1065/2104] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:18:43 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:18:44.080685 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1040/2105] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:18:43 2022] PUT /identity/v3/projects/1cdc0123b9f34602ab3aeac846b099f6/users/b149cd4e47bc4e109505b5422856bdb9/roles/4eea39e24b06431aa357053fe7907a3d => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:18:44.106206 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fbbe6b-bf88-4fba-81a2-77569c2ef674 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:18:44.107143 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fbbe6b-bf88-4fba-81a2-77569c2ef674 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:18:44.108042 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fbbe6b-bf88-4fba-81a2-77569c2ef674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:18:44.190700 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-52fbbe6b-bf88-4fba-81a2-77569c2ef674 None None] MFA Rules not processed for user `b149cd4e47bc4e109505b5422856bdb9`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:18:44.192201 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-52fbbe6b-bf88-4fba-81a2-77569c2ef674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:18:44.199331 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef51d26-4477-4596-bf46-d92798494187 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:18:44.199787 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef51d26-4477-4596-bf46-d92798494187 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:18:44.200155 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef51d26-4477-4596-bf46-d92798494187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:18:44.241088 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-1ef51d26-4477-4596-bf46-d92798494187 None None] MFA Rules not processed for user `aaa686e692444eae835ca343926e0595`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:18:44.242540 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-1ef51d26-4477-4596-bf46-d92798494187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:18:44.426373 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1041/2106] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:18:44 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:18:44.446986 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1066/2107] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:18:44 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 344 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:18:44.469343 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef51d26-4477-4596-bf46-d92798494187 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:18:44.504161 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:18:44.710110 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef51d26-4477-4596-bf46-d92798494187 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:18:44.751067 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca15b97-685b-4fd1-aa44-d5b61d135b11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:18:44.756096 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca15b97-685b-4fd1-aa44-d5b61d135b11 admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:18:44.758463 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca15b97-685b-4fd1-aa44-d5b61d135b11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:18:44.759166 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca15b97-685b-4fd1-aa44-d5b61d135b11 admin admin] PATH_INFO: `/v3/projects` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:18:44.762475 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca15b97-685b-4fd1-aa44-d5b61d135b11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:18:44.779527 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca15b97-685b-4fd1-aa44-d5b61d135b11 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:18:44.940119 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1042/2108] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:18:44 2022] POST /identity/v3/projects => generated 369 bytes in 475 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:18:44.947742 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fbbe6b-bf88-4fba-81a2-77569c2ef674 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:18:44.983725 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fbbe6b-bf88-4fba-81a2-77569c2ef674 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:18:45.111716 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:18:45.229377 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ca4586-7df3-4499-939a-beca35cce234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:18:45.230625 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ca4586-7df3-4499-939a-beca35cce234 admin admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:18:45.231124 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ca4586-7df3-4499-939a-beca35cce234 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:18:45.231615 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ca4586-7df3-4499-939a-beca35cce234 admin admin] PATH_INFO: `/v3/users` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:18:45.233065 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ca4586-7df3-4499-939a-beca35cce234 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:18:45.249821 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ca4586-7df3-4499-939a-beca35cce234 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:18:45.315033 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1067/2109] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:18:44 2022] POST /identity/v3/users => generated 291 bytes in 364 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:18:45.317354 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca15b97-685b-4fd1-aa44-d5b61d135b11 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:18:45.336752 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca15b97-685b-4fd1-aa44-d5b61d135b11 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:18:45.371101 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa21e49a-5713-4c5b-bf73-55da32630694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:18:45.371864 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa21e49a-5713-4c5b-bf73-55da32630694 admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:18:45.372374 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa21e49a-5713-4c5b-bf73-55da32630694 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:18:45.372867 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa21e49a-5713-4c5b-bf73-55da32630694 admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:18:45.379089 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa21e49a-5713-4c5b-bf73-55da32630694 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:18:45.387469 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa21e49a-5713-4c5b-bf73-55da32630694 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:18:45.410206 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1043/2110] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:18:45 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 20:18:45.416188 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ca4586-7df3-4499-939a-beca35cce234 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:18:45.432142 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ca4586-7df3-4499-939a-beca35cce234 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:18:45.474066 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b543a130-d970-4f66-aaa8-318f34e8cebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:18:45.475078 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b543a130-d970-4f66-aaa8-318f34e8cebd admin admin] REQUEST_METHOD: `PUT` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:18:45.475532 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b543a130-d970-4f66-aaa8-318f34e8cebd admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:18:45.475929 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b543a130-d970-4f66-aaa8-318f34e8cebd admin admin] PATH_INFO: `/v3/projects/9465ec20a0b94441b4d16e6c3aa223eb/users/d3d6efb0d50940f08ae1082958e1d2d1/roles/e0ab41d8ae22474688b5361bcc75e0f0` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:18:45.538024 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b543a130-d970-4f66-aaa8-318f34e8cebd admin admin] RBAC: Authorizing `identity:create_grant(project_id=9465ec20a0b94441b4d16e6c3aa223eb, user_id=d3d6efb0d50940f08ae1082958e1d2d1, role_id=e0ab41d8ae22474688b5361bcc75e0f0)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:18:45.578780 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b543a130-d970-4f66-aaa8-318f34e8cebd admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:18:45.658951 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1068/2111] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:18:45 2022] PUT /identity/v3/projects/9465ec20a0b94441b4d16e6c3aa223eb/users/d3d6efb0d50940f08ae1082958e1d2d1/roles/e0ab41d8ae22474688b5361bcc75e0f0 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:18:45.665074 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa21e49a-5713-4c5b-bf73-55da32630694 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:18:45.712254 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa21e49a-5713-4c5b-bf73-55da32630694 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:18:45.994775 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d480dd-1927-49f8-ba72-25c8a8333f3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:18:45.995810 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d480dd-1927-49f8-ba72-25c8a8333f3c admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:18:45.996448 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d480dd-1927-49f8-ba72-25c8a8333f3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:18:45.996945 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d480dd-1927-49f8-ba72-25c8a8333f3c admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:18:45.998228 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d480dd-1927-49f8-ba72-25c8a8333f3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:18:46.011301 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d480dd-1927-49f8-ba72-25c8a8333f3c admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:18:46.064143 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1044/2112] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:18:45 2022] GET /identity/v3/roles => generated 1420 bytes in 400 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:18:46.070813 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b543a130-d970-4f66-aaa8-318f34e8cebd admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:18:46.097861 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b543a130-d970-4f66-aaa8-318f34e8cebd admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:18:46.106888 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4da574-6f82-42ec-9ffe-3b73fb10a3ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:18:46.108157 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4da574-6f82-42ec-9ffe-3b73fb10a3ce admin admin] REQUEST_METHOD: `PUT` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:18:46.108498 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4da574-6f82-42ec-9ffe-3b73fb10a3ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:18:46.109744 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4da574-6f82-42ec-9ffe-3b73fb10a3ce admin admin] PATH_INFO: `/v3/projects/9465ec20a0b94441b4d16e6c3aa223eb/users/d3d6efb0d50940f08ae1082958e1d2d1/roles/e0ab41d8ae22474688b5361bcc75e0f0` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:18:46.113500 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4da574-6f82-42ec-9ffe-3b73fb10a3ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=9465ec20a0b94441b4d16e6c3aa223eb, user_id=d3d6efb0d50940f08ae1082958e1d2d1, role_id=e0ab41d8ae22474688b5361bcc75e0f0)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:18:46.119206 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4da574-6f82-42ec-9ffe-3b73fb10a3ce admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:18:46.166077 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1069/2113] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:18:46 2022] PUT /identity/v3/projects/9465ec20a0b94441b4d16e6c3aa223eb/users/d3d6efb0d50940f08ae1082958e1d2d1/roles/e0ab41d8ae22474688b5361bcc75e0f0 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:18:46.180544 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e1a89e-e8cc-420c-b3b8-4d453fe0c7ba None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:18:46.186441 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e1a89e-e8cc-420c-b3b8-4d453fe0c7ba None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:18:46.186856 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e1a89e-e8cc-420c-b3b8-4d453fe0c7ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:18:46.259126 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-62e1a89e-e8cc-420c-b3b8-4d453fe0c7ba None None] MFA Rules not processed for user `d3d6efb0d50940f08ae1082958e1d2d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:18:46.266666 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-62e1a89e-e8cc-420c-b3b8-4d453fe0c7ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:18:46.483663 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1045/2114] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:18:46 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:18:46.498500 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4da574-6f82-42ec-9ffe-3b73fb10a3ce admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:18:46.502218 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-5b4da574-6f82-42ec-9ffe-3b73fb10a3ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:18:46.688236 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4da574-6f82-42ec-9ffe-3b73fb10a3ce admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:18:46.700954 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e217ac5-c7f3-45e2-ac17-5d901a85fc15 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df1bca993bc142908cc7cf85eabd2b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:18:46.702152 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e217ac5-c7f3-45e2-ac17-5d901a85fc15 service cinder] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:18:46.702553 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e217ac5-c7f3-45e2-ac17-5d901a85fc15 service cinder] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:18:46.703844 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e217ac5-c7f3-45e2-ac17-5d901a85fc15 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:18:46.736427 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e217ac5-c7f3-45e2-ac17-5d901a85fc15 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:18:46.741974 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e217ac5-c7f3-45e2-ac17-5d901a85fc15 service cinder] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:18:46.778205 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1070/2115] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:18:46 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:18:49.841805 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e1a89e-e8cc-420c-b3b8-4d453fe0c7ba None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:18:49.855151 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:18:50.008056 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e1a89e-e8cc-420c-b3b8-4d453fe0c7ba None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:18:50.013780 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92e089c-b94a-4c8b-9aab-21cc3056d612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:18:50.014602 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92e089c-b94a-4c8b-9aab-21cc3056d612 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:18:50.014903 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92e089c-b94a-4c8b-9aab-21cc3056d612 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:18:50.015779 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92e089c-b94a-4c8b-9aab-21cc3056d612 admin admin] PATH_INFO: `/v3/users/b149cd4e47bc4e109505b5422856bdb9` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:18:50.017109 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92e089c-b94a-4c8b-9aab-21cc3056d612 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b149cd4e47bc4e109505b5422856bdb9)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:18:50.019382 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92e089c-b94a-4c8b-9aab-21cc3056d612 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:18:50.134001 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-c92e089c-b94a-4c8b-9aab-21cc3056d612 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b149cd4e47bc4e109505b5422856bdb9'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:18:50.169675 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-c92e089c-b94a-4c8b-9aab-21cc3056d612 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b149cd4e47bc4e109505b5422856bdb9'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:18:50.198434 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-c92e089c-b94a-4c8b-9aab-21cc3056d612 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b149cd4e47bc4e109505b5422856bdb9'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:18:50.199799 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-c92e089c-b94a-4c8b-9aab-21cc3056d612 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b149cd4e47bc4e109505b5422856bdb9'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:18:50.220436 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-c92e089c-b94a-4c8b-9aab-21cc3056d612 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b149cd4e47bc4e109505b5422856bdb9'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:18:50.259268 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1046/2116] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:18:49 2022] DELETE /identity/v3/users/b149cd4e47bc4e109505b5422856bdb9 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:18:50.268571 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc6bcb7-a327-46ba-88f5-72a4eadfc3e1 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:18:50.269346 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc6bcb7-a327-46ba-88f5-72a4eadfc3e1 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:18:50.269559 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc6bcb7-a327-46ba-88f5-72a4eadfc3e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:18:50.307738 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-0cc6bcb7-a327-46ba-88f5-72a4eadfc3e1 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:18:50.309214 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-0cc6bcb7-a327-46ba-88f5-72a4eadfc3e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:18:50.438855 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1071/2117] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:18:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:18:50.454621 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92e089c-b94a-4c8b-9aab-21cc3056d612 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:18:50.456540 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-c92e089c-b94a-4c8b-9aab-21cc3056d612 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:18:50.558302 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92e089c-b94a-4c8b-9aab-21cc3056d612 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:18:50.569451 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a361d0fb-5ab4-4b9c-a197-ce4f5ca51293 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:18:50.570370 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a361d0fb-5ab4-4b9c-a197-ce4f5ca51293 service neutron] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:18:50.570809 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a361d0fb-5ab4-4b9c-a197-ce4f5ca51293 service neutron] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:18:50.571289 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a361d0fb-5ab4-4b9c-a197-ce4f5ca51293 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:18:50.584722 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a361d0fb-5ab4-4b9c-a197-ce4f5ca51293 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:18:50.586389 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a361d0fb-5ab4-4b9c-a197-ce4f5ca51293 service neutron] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:18:50.606962 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1047/2118] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:18:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 20:18:51.022687 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc6bcb7-a327-46ba-88f5-72a4eadfc3e1 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:18:51.025200 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-0cc6bcb7-a327-46ba-88f5-72a4eadfc3e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:18:51.047366 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc6bcb7-a327-46ba-88f5-72a4eadfc3e1 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:18:51.059147 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b61af7-c205-48c2-a839-39b1391df0ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:18:51.059147 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b61af7-c205-48c2-a839-39b1391df0ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:18:51.059147 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b61af7-c205-48c2-a839-39b1391df0ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:18:51.059810 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b61af7-c205-48c2-a839-39b1391df0ef admin admin] PATH_INFO: `/v3/projects/1cdc0123b9f34602ab3aeac846b099f6` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:18:51.063017 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b61af7-c205-48c2-a839-39b1391df0ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=1cdc0123b9f34602ab3aeac846b099f6)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:18:51.068787 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b61af7-c205-48c2-a839-39b1391df0ef admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:18:51.111735 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:18:51.167461 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-14b61af7-c205-48c2-a839-39b1391df0ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1cdc0123b9f34602ab3aeac846b099f6', 'request_id': 'req-14b61af7-c205-48c2-a839-39b1391df0ef'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:18:51.181212 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-14b61af7-c205-48c2-a839-39b1391df0ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1cdc0123b9f34602ab3aeac846b099f6', 'request_id': 'req-14b61af7-c205-48c2-a839-39b1391df0ef'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:18:51.190713 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-14b61af7-c205-48c2-a839-39b1391df0ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1cdc0123b9f34602ab3aeac846b099f6', 'request_id': 'req-14b61af7-c205-48c2-a839-39b1391df0ef'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:18:51.217357 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-14b61af7-c205-48c2-a839-39b1391df0ef admin admin] The token cache is being invalidate because project 1cdc0123b9f34602ab3aeac846b099f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81772) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:18:51.217972 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-14b61af7-c205-48c2-a839-39b1391df0ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:18:51.219697 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-14b61af7-c205-48c2-a839-39b1391df0ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:18:51.223220 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1072/2119] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:18:51 2022] DELETE /identity/v3/projects/1cdc0123b9f34602ab3aeac846b099f6 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:18:51.228678 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a361d0fb-5ab4-4b9c-a197-ce4f5ca51293 service neutron] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:18:51.262348 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a361d0fb-5ab4-4b9c-a197-ce4f5ca51293 service neutron] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:18:51.290326 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:18:51.366990 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835666b2-f760-4f3a-9228-5a00e0b0bb72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:18:51.368172 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835666b2-f760-4f3a-9228-5a00e0b0bb72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:18:51.369428 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835666b2-f760-4f3a-9228-5a00e0b0bb72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:18:51.369428 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835666b2-f760-4f3a-9228-5a00e0b0bb72 admin admin] PATH_INFO: `/v3/users/d3d6efb0d50940f08ae1082958e1d2d1` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:18:51.372254 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835666b2-f760-4f3a-9228-5a00e0b0bb72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3d6efb0d50940f08ae1082958e1d2d1)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:18:51.378080 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835666b2-f760-4f3a-9228-5a00e0b0bb72 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:18:51.468571 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-835666b2-f760-4f3a-9228-5a00e0b0bb72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3d6efb0d50940f08ae1082958e1d2d1'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:18:51.480903 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-835666b2-f760-4f3a-9228-5a00e0b0bb72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3d6efb0d50940f08ae1082958e1d2d1'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:18:51.481936 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-835666b2-f760-4f3a-9228-5a00e0b0bb72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3d6efb0d50940f08ae1082958e1d2d1'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:18:51.482565 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-835666b2-f760-4f3a-9228-5a00e0b0bb72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3d6efb0d50940f08ae1082958e1d2d1'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:18:51.495517 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-835666b2-f760-4f3a-9228-5a00e0b0bb72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3d6efb0d50940f08ae1082958e1d2d1'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:18:51.536238 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1048/2120] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:18:51 2022] DELETE /identity/v3/users/d3d6efb0d50940f08ae1082958e1d2d1 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 20:18:51.967503 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b61af7-c205-48c2-a839-39b1391df0ef admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:18:51.969649 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-14b61af7-c205-48c2-a839-39b1391df0ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:18:51.986718 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b61af7-c205-48c2-a839-39b1391df0ef admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:18:52.087866 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cfaca8-0eb5-420f-a7f9-ce91f3745c90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:18:52.088696 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cfaca8-0eb5-420f-a7f9-ce91f3745c90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:18:52.089137 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cfaca8-0eb5-420f-a7f9-ce91f3745c90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:18:52.089307 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cfaca8-0eb5-420f-a7f9-ce91f3745c90 admin admin] PATH_INFO: `/v3/projects/9465ec20a0b94441b4d16e6c3aa223eb` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:18:52.090706 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cfaca8-0eb5-420f-a7f9-ce91f3745c90 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9465ec20a0b94441b4d16e6c3aa223eb)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:18:52.092888 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cfaca8-0eb5-420f-a7f9-ce91f3745c90 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:18:52.162885 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-b0cfaca8-0eb5-420f-a7f9-ce91f3745c90 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9465ec20a0b94441b4d16e6c3aa223eb', 'request_id': 'req-b0cfaca8-0eb5-420f-a7f9-ce91f3745c90'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:18:52.175090 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-b0cfaca8-0eb5-420f-a7f9-ce91f3745c90 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9465ec20a0b94441b4d16e6c3aa223eb', 'request_id': 'req-b0cfaca8-0eb5-420f-a7f9-ce91f3745c90'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:18:52.183811 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-b0cfaca8-0eb5-420f-a7f9-ce91f3745c90 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9465ec20a0b94441b4d16e6c3aa223eb', 'request_id': 'req-b0cfaca8-0eb5-420f-a7f9-ce91f3745c90'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:18:52.196154 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-b0cfaca8-0eb5-420f-a7f9-ce91f3745c90 admin admin] The token cache is being invalidate because project 9465ec20a0b94441b4d16e6c3aa223eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81772) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:18:52.196534 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-b0cfaca8-0eb5-420f-a7f9-ce91f3745c90 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:18:52.197518 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-b0cfaca8-0eb5-420f-a7f9-ce91f3745c90 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:18:52.199126 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1073/2121] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:18:51 2022] DELETE /identity/v3/projects/9465ec20a0b94441b4d16e6c3aa223eb => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:18:52.205967 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8430f7e5-f1a2-420f-8c11-c17f4588302e None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:18:52.206521 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8430f7e5-f1a2-420f-8c11-c17f4588302e None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:18:52.206884 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8430f7e5-f1a2-420f-8c11-c17f4588302e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:18:52.231302 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-8430f7e5-f1a2-420f-8c11-c17f4588302e None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:18:52.232015 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-8430f7e5-f1a2-420f-8c11-c17f4588302e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:18:52.332114 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1049/2122] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:18:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:18:52.344968 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbee76e9-e215-4bea-9afa-038cdda4eb32 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:18:52.345997 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbee76e9-e215-4bea-9afa-038cdda4eb32 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:18:52.346428 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbee76e9-e215-4bea-9afa-038cdda4eb32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:18:52.398265 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-dbee76e9-e215-4bea-9afa-038cdda4eb32 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:18:52.403722 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-dbee76e9-e215-4bea-9afa-038cdda4eb32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:18:52.431267 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1074/2123] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:18:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:18:52.438101 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8430f7e5-f1a2-420f-8c11-c17f4588302e None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:18:52.451259 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8430f7e5-f1a2-420f-8c11-c17f4588302e None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:18:52.458623 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c065b521-14a1-4820-88a7-afe7b797f756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:18:52.460382 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c065b521-14a1-4820-88a7-afe7b797f756 admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:18:52.460382 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c065b521-14a1-4820-88a7-afe7b797f756 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:18:52.460837 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c065b521-14a1-4820-88a7-afe7b797f756 admin admin] PATH_INFO: `/v3/domains` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:18:52.462343 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c065b521-14a1-4820-88a7-afe7b797f756 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:18:52.463155 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c065b521-14a1-4820-88a7-afe7b797f756 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:18:52.465225 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c065b521-14a1-4820-88a7-afe7b797f756 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:18:52.499945 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1050/2124] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 6 20:18:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:18:52.502455 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbee76e9-e215-4bea-9afa-038cdda4eb32 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:18:52.522227 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbee76e9-e215-4bea-9afa-038cdda4eb32 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:18:52.533091 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9a2528-3e29-4c70-85ae-8cb5b2a10974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:18:52.534297 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9a2528-3e29-4c70-85ae-8cb5b2a10974 admin admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:18:52.534803 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9a2528-3e29-4c70-85ae-8cb5b2a10974 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:18:52.535292 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9a2528-3e29-4c70-85ae-8cb5b2a10974 admin admin] PATH_INFO: `/v3/projects` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:18:52.536546 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9a2528-3e29-4c70-85ae-8cb5b2a10974 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:18:52.540835 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9a2528-3e29-4c70-85ae-8cb5b2a10974 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:18:52.635314 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1075/2125] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:18:52 2022] POST /identity/v3/projects => generated 383 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:18:52.639557 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c065b521-14a1-4820-88a7-afe7b797f756 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:18:52.672047 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c065b521-14a1-4820-88a7-afe7b797f756 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:18:52.757963 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:18:52.829195 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd9d518-15e5-470e-a669-84a73b48c876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:18:52.830874 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd9d518-15e5-470e-a669-84a73b48c876 admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:18:52.831464 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd9d518-15e5-470e-a669-84a73b48c876 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:18:52.831965 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd9d518-15e5-470e-a669-84a73b48c876 admin admin] PATH_INFO: `/v3/users` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:18:52.833935 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd9d518-15e5-470e-a669-84a73b48c876 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:18:52.840387 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd9d518-15e5-470e-a669-84a73b48c876 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:18:52.871675 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1051/2126] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:18:52 2022] POST /identity/v3/users => generated 292 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:18:52.878902 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9a2528-3e29-4c70-85ae-8cb5b2a10974 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:18:52.899466 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9a2528-3e29-4c70-85ae-8cb5b2a10974 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:18:52.907326 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d465a5-7697-45b3-ad10-a64b61d689d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:18:52.908285 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d465a5-7697-45b3-ad10-a64b61d689d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:18:52.908620 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d465a5-7697-45b3-ad10-a64b61d689d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:18:52.908938 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d465a5-7697-45b3-ad10-a64b61d689d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:18:52.909917 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d465a5-7697-45b3-ad10-a64b61d689d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:18:52.911801 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d465a5-7697-45b3-ad10-a64b61d689d1 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:18:52.928079 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1076/2127] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:18:52 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:18:52.936138 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd9d518-15e5-470e-a669-84a73b48c876 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:18:52.964654 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd9d518-15e5-470e-a669-84a73b48c876 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:18:52.973807 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82a4cce-2814-446a-8f38-672d30044fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:18:52.974917 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82a4cce-2814-446a-8f38-672d30044fdd admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:18:52.975718 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82a4cce-2814-446a-8f38-672d30044fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:18:52.976300 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82a4cce-2814-446a-8f38-672d30044fdd admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:18:52.977633 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82a4cce-2814-446a-8f38-672d30044fdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:18:52.980939 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82a4cce-2814-446a-8f38-672d30044fdd admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:18:53.009807 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1052/2128] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:18:52 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:18:53.011771 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d465a5-7697-45b3-ad10-a64b61d689d1 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:18:53.019314 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:18:53.028661 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d465a5-7697-45b3-ad10-a64b61d689d1 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:18:53.041562 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5218d04f-98c7-4d34-b0dc-fee9c8224dad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:18:53.042910 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5218d04f-98c7-4d34-b0dc-fee9c8224dad admin admin] REQUEST_METHOD: `PUT` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:18:53.043256 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5218d04f-98c7-4d34-b0dc-fee9c8224dad admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:18:53.043744 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5218d04f-98c7-4d34-b0dc-fee9c8224dad admin admin] PATH_INFO: `/v3/projects/58fce450291d42d2a662047f5d5ffea9/users/cbb7ba10947748a6bd76ec1780551c03/roles/4eea39e24b06431aa357053fe7907a3d` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:18:53.096339 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5218d04f-98c7-4d34-b0dc-fee9c8224dad admin admin] RBAC: Authorizing `identity:create_grant(project_id=58fce450291d42d2a662047f5d5ffea9, user_id=cbb7ba10947748a6bd76ec1780551c03, role_id=4eea39e24b06431aa357053fe7907a3d)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:18:53.101854 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5218d04f-98c7-4d34-b0dc-fee9c8224dad admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:18:53.132533 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1077/2129] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:18:53 2022] PUT /identity/v3/projects/58fce450291d42d2a662047f5d5ffea9/users/cbb7ba10947748a6bd76ec1780551c03/roles/4eea39e24b06431aa357053fe7907a3d => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:18:53.151243 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0ce93b-d4f4-47d0-92f2-975d9104d2c7 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:18:53.153641 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0ce93b-d4f4-47d0-92f2-975d9104d2c7 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:18:53.153860 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0ce93b-d4f4-47d0-92f2-975d9104d2c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:18:53.204188 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-2c0ce93b-d4f4-47d0-92f2-975d9104d2c7 None None] MFA Rules not processed for user `cbb7ba10947748a6bd76ec1780551c03`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:18:53.207266 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-2c0ce93b-d4f4-47d0-92f2-975d9104d2c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:18:53.404830 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1053/2130] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:18:53 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:18:53.414631 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5218d04f-98c7-4d34-b0dc-fee9c8224dad admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:18:53.617663 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5218d04f-98c7-4d34-b0dc-fee9c8224dad admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:18:53.639038 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9735882a-daa3-443b-a4fe-a51cafc98fbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:18:53.640787 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9735882a-daa3-443b-a4fe-a51cafc98fbf admin admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:18:53.641664 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9735882a-daa3-443b-a4fe-a51cafc98fbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:18:53.642083 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9735882a-daa3-443b-a4fe-a51cafc98fbf admin admin] PATH_INFO: `/v3/projects` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:18:53.643797 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9735882a-daa3-443b-a4fe-a51cafc98fbf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:18:53.648302 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9735882a-daa3-443b-a4fe-a51cafc98fbf admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:18:53.722344 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1078/2131] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:18:53 2022] POST /identity/v3/projects => generated 385 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:18:53.723984 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0ce93b-d4f4-47d0-92f2-975d9104d2c7 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:18:53.746113 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0ce93b-d4f4-47d0-92f2-975d9104d2c7 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:18:53.948403 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad2fb73-4a8e-4c3c-bf49-48ad35885504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:18:53.952494 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad2fb73-4a8e-4c3c-bf49-48ad35885504 admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:18:53.952494 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad2fb73-4a8e-4c3c-bf49-48ad35885504 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:18:53.952494 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad2fb73-4a8e-4c3c-bf49-48ad35885504 admin admin] PATH_INFO: `/v3/users` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:18:53.952494 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad2fb73-4a8e-4c3c-bf49-48ad35885504 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:18:53.955929 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad2fb73-4a8e-4c3c-bf49-48ad35885504 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:18:54.006507 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1054/2132] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:18:53 2022] POST /identity/v3/users => generated 299 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 20:18:54.011805 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9735882a-daa3-443b-a4fe-a51cafc98fbf admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:18:54.028071 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9735882a-daa3-443b-a4fe-a51cafc98fbf admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:18:54.047587 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ac44d1-5d57-413f-a051-b75f24f52c2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:18:54.048772 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ac44d1-5d57-413f-a051-b75f24f52c2e admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:18:54.049100 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ac44d1-5d57-413f-a051-b75f24f52c2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:18:54.049565 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ac44d1-5d57-413f-a051-b75f24f52c2e admin admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:18:54.050851 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ac44d1-5d57-413f-a051-b75f24f52c2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:18:54.058862 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ac44d1-5d57-413f-a051-b75f24f52c2e admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:18:54.082004 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1079/2133] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:18:54 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:18:54.086566 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad2fb73-4a8e-4c3c-bf49-48ad35885504 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:18:54.100984 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad2fb73-4a8e-4c3c-bf49-48ad35885504 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:18:54.107559 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b99118-d01b-477e-a271-e4374c769610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:18:54.108987 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b99118-d01b-477e-a271-e4374c769610 admin admin] REQUEST_METHOD: `PUT` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:18:54.109404 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b99118-d01b-477e-a271-e4374c769610 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:18:54.109829 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b99118-d01b-477e-a271-e4374c769610 admin admin] PATH_INFO: `/v3/projects/42841ec83d874b6e84c77b6de1b9a3c8/users/00d9ad180cee4ce997a5f325ac3cea8a/roles/e0ab41d8ae22474688b5361bcc75e0f0` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:18:54.144154 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b99118-d01b-477e-a271-e4374c769610 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42841ec83d874b6e84c77b6de1b9a3c8, user_id=00d9ad180cee4ce997a5f325ac3cea8a, role_id=e0ab41d8ae22474688b5361bcc75e0f0)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:18:54.147255 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b99118-d01b-477e-a271-e4374c769610 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:18:54.174174 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1055/2134] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:18:54 2022] PUT /identity/v3/projects/42841ec83d874b6e84c77b6de1b9a3c8/users/00d9ad180cee4ce997a5f325ac3cea8a/roles/e0ab41d8ae22474688b5361bcc75e0f0 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:18:54.174995 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ac44d1-5d57-413f-a051-b75f24f52c2e admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:18:54.190605 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ac44d1-5d57-413f-a051-b75f24f52c2e admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:18:54.298874 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b5cbd3-77e0-4327-b8d5-c32bb0b3f18c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:18:54.301433 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b5cbd3-77e0-4327-b8d5-c32bb0b3f18c admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:18:54.302786 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b5cbd3-77e0-4327-b8d5-c32bb0b3f18c admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:18:54.303741 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b5cbd3-77e0-4327-b8d5-c32bb0b3f18c admin admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:18:54.305550 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b5cbd3-77e0-4327-b8d5-c32bb0b3f18c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:18:54.308902 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b5cbd3-77e0-4327-b8d5-c32bb0b3f18c admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:18:54.382816 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1080/2135] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:18:54 2022] GET /identity/v3/roles => generated 1420 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:18:54.388009 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b99118-d01b-477e-a271-e4374c769610 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:18:54.408222 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:18:54.427549 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b99118-d01b-477e-a271-e4374c769610 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:18:54.449260 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2794bfb-5e52-493a-ab9e-5f76c340d328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:18:54.451132 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2794bfb-5e52-493a-ab9e-5f76c340d328 admin admin] REQUEST_METHOD: `PUT` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:18:54.451891 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2794bfb-5e52-493a-ab9e-5f76c340d328 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:18:54.453210 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2794bfb-5e52-493a-ab9e-5f76c340d328 admin admin] PATH_INFO: `/v3/projects/42841ec83d874b6e84c77b6de1b9a3c8/users/00d9ad180cee4ce997a5f325ac3cea8a/roles/e0ab41d8ae22474688b5361bcc75e0f0` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:18:54.457438 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2794bfb-5e52-493a-ab9e-5f76c340d328 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42841ec83d874b6e84c77b6de1b9a3c8, user_id=00d9ad180cee4ce997a5f325ac3cea8a, role_id=e0ab41d8ae22474688b5361bcc75e0f0)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:18:54.463765 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2794bfb-5e52-493a-ab9e-5f76c340d328 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:18:54.489574 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1056/2136] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:18:54 2022] PUT /identity/v3/projects/42841ec83d874b6e84c77b6de1b9a3c8/users/00d9ad180cee4ce997a5f325ac3cea8a/roles/e0ab41d8ae22474688b5361bcc75e0f0 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:18:54.507680 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86395840-e6d2-4fd4-9a21-c49fc4dc9650 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:18:54.509902 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86395840-e6d2-4fd4-9a21-c49fc4dc9650 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:18:54.510765 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86395840-e6d2-4fd4-9a21-c49fc4dc9650 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:18:54.612245 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-86395840-e6d2-4fd4-9a21-c49fc4dc9650 None None] MFA Rules not processed for user `00d9ad180cee4ce997a5f325ac3cea8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:18:54.617431 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-86395840-e6d2-4fd4-9a21-c49fc4dc9650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:18:54.687782 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:18:54.743558 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1081/2137] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:18:54 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:18:54.769281 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2794bfb-5e52-493a-ab9e-5f76c340d328 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:18:54.771692 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-a2794bfb-5e52-493a-ab9e-5f76c340d328 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:18:54.906666 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2794bfb-5e52-493a-ab9e-5f76c340d328 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:18:54.922618 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949117d8-4840-4aae-ac6d-6037a5c2e38f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df1bca993bc142908cc7cf85eabd2b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:18:54.924333 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949117d8-4840-4aae-ac6d-6037a5c2e38f service cinder] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:18:54.924999 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949117d8-4840-4aae-ac6d-6037a5c2e38f service cinder] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:18:54.925454 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949117d8-4840-4aae-ac6d-6037a5c2e38f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:18:55.038159 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949117d8-4840-4aae-ac6d-6037a5c2e38f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:18:55.041914 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949117d8-4840-4aae-ac6d-6037a5c2e38f service cinder] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:18:55.055447 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1057/2138] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:18:54 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 20:18:55.264692 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb02016-cfe1-4d9a-a808-d01b695bebd2 None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:18:55.265995 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb02016-cfe1-4d9a-a808-d01b695bebd2 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:18:55.266140 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb02016-cfe1-4d9a-a808-d01b695bebd2 None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:18:55.268526 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1082/2139] 10.222.0.20 () {54 vars in 921 bytes} [Tue Sep 6 20:18:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 20:18:55.281848 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0cb5e1-5e4d-48e7-b0b0-57d807e59c07 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:18:55.282523 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0cb5e1-5e4d-48e7-b0b0-57d807e59c07 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:18:55.282865 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0cb5e1-5e4d-48e7-b0b0-57d807e59c07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:18:55.315901 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-9b0cb5e1-5e4d-48e7-b0b0-57d807e59c07 None None] MFA Rules not processed for user `5776cbf4f6714b2083c3ebdd6d98cc87`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:18:55.317363 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-9b0cb5e1-5e4d-48e7-b0b0-57d807e59c07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:18:55.462333 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1058/2140] 10.222.0.20 () {58 vars in 1018 bytes} [Tue Sep 6 20:18:55 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:18:58.553414 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422d2e9c-c663-4f60-92b3-de84674d8d35 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:18:58.554151 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422d2e9c-c663-4f60-92b3-de84674d8d35 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:18:58.554151 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422d2e9c-c663-4f60-92b3-de84674d8d35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:18:58.576545 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-422d2e9c-c663-4f60-92b3-de84674d8d35 None None] MFA Rules not processed for user `87f8cc79586f4cb79c962ede8342f580`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:18:58.578075 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-422d2e9c-c663-4f60-92b3-de84674d8d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:18:58.650545 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1083/2141] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:18:58 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:19:02.151749 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0cb5e1-5e4d-48e7-b0b0-57d807e59c07 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:19:02.154517 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-9b0cb5e1-5e4d-48e7-b0b0-57d807e59c07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:19:02.176892 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0cb5e1-5e4d-48e7-b0b0-57d807e59c07 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:19:02.183425 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d45d110-9023-46bc-971f-c4da330b4dec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df1bca993bc142908cc7cf85eabd2b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:19:02.184471 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d45d110-9023-46bc-971f-c4da330b4dec service cinder] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:19:02.184975 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d45d110-9023-46bc-971f-c4da330b4dec service cinder] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:19:02.185469 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d45d110-9023-46bc-971f-c4da330b4dec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:19:02.203210 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d45d110-9023-46bc-971f-c4da330b4dec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:19:02.204768 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d45d110-9023-46bc-971f-c4da330b4dec service cinder] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:19:02.217646 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1059/2142] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:19:02 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:19:07.183270 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422d2e9c-c663-4f60-92b3-de84674d8d35 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:19:07.184787 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-422d2e9c-c663-4f60-92b3-de84674d8d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:19:07.253017 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422d2e9c-c663-4f60-92b3-de84674d8d35 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:19:07.258175 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf0fe73-196c-46df-af07-ea23978b2895 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07af18f35ffc42a295278d5349e06381', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:19:07.258812 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf0fe73-196c-46df-af07-ea23978b2895 service glance] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:19:07.259092 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf0fe73-196c-46df-af07-ea23978b2895 service glance] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:19:07.259509 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf0fe73-196c-46df-af07-ea23978b2895 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:19:07.271334 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf0fe73-196c-46df-af07-ea23978b2895 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:19:07.272804 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf0fe73-196c-46df-af07-ea23978b2895 service glance] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:19:07.287026 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1084/2143] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:19:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 20:19:07.434904 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e840005-6e3f-4f4c-be91-6b41f439ca8d None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:19:07.437034 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e840005-6e3f-4f4c-be91-6b41f439ca8d None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:19:07.438174 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e840005-6e3f-4f4c-be91-6b41f439ca8d None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:19:07.442319 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1060/2144] 10.222.0.20 () {54 vars in 921 bytes} [Tue Sep 6 20:19:07 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:19:07.449564 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf0fe73-196c-46df-af07-ea23978b2895 service glance] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:19:07.452986 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850076a7-1aad-40cf-9ff0-698663a8701f None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:19:07.453450 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850076a7-1aad-40cf-9ff0-698663a8701f None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:19:07.453947 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850076a7-1aad-40cf-9ff0-698663a8701f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:19:07.488929 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-850076a7-1aad-40cf-9ff0-698663a8701f None None] MFA Rules not processed for user `32882d38295046fea004a659a71bde58`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:19:07.491545 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-850076a7-1aad-40cf-9ff0-698663a8701f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:19:07.525196 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:19:07.595842 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1061/2145] 10.222.0.20 () {58 vars in 1018 bytes} [Tue Sep 6 20:19:07 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:19:07.603890 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf0fe73-196c-46df-af07-ea23978b2895 service glance] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:19:07.615701 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6352aeea-29f2-4b70-819b-b6f0cff87222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:19:07.615701 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6352aeea-29f2-4b70-819b-b6f0cff87222 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:19:07.615701 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6352aeea-29f2-4b70-819b-b6f0cff87222 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:19:07.616427 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6352aeea-29f2-4b70-819b-b6f0cff87222 admin admin] PATH_INFO: `/v3/users/cbb7ba10947748a6bd76ec1780551c03` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:19:07.617382 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850076a7-1aad-40cf-9ff0-698663a8701f None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:19:07.619871 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-850076a7-1aad-40cf-9ff0-698663a8701f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:19:07.623388 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6352aeea-29f2-4b70-819b-b6f0cff87222 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbb7ba10947748a6bd76ec1780551c03)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:19:07.631324 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6352aeea-29f2-4b70-819b-b6f0cff87222 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:19:07.645059 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850076a7-1aad-40cf-9ff0-698663a8701f None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:19:07.653795 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b424d8-fcdf-470d-a0e4-5ed7822aa3f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:19:07.655155 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b424d8-fcdf-470d-a0e4-5ed7822aa3f5 service neutron] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:19:07.656397 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b424d8-fcdf-470d-a0e4-5ed7822aa3f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:19:07.657524 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b424d8-fcdf-470d-a0e4-5ed7822aa3f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:19:07.676694 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b424d8-fcdf-470d-a0e4-5ed7822aa3f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:19:07.680683 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b424d8-fcdf-470d-a0e4-5ed7822aa3f5 service neutron] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:19:07.700709 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:19:07.709589 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1062/2146] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:19:07 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 20:19:07.771145 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-6352aeea-29f2-4b70-819b-b6f0cff87222 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbb7ba10947748a6bd76ec1780551c03'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:19:07.772320 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-6352aeea-29f2-4b70-819b-b6f0cff87222 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbb7ba10947748a6bd76ec1780551c03'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:19:07.794636 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-6352aeea-29f2-4b70-819b-b6f0cff87222 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbb7ba10947748a6bd76ec1780551c03'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:19:07.797303 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-6352aeea-29f2-4b70-819b-b6f0cff87222 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbb7ba10947748a6bd76ec1780551c03'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:19:07.819638 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-6352aeea-29f2-4b70-819b-b6f0cff87222 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbb7ba10947748a6bd76ec1780551c03'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:19:07.840200 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1085/2147] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:19:07 2022] DELETE /identity/v3/users/cbb7ba10947748a6bd76ec1780551c03 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:19:07.847412 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aa2c57-25bc-49b2-9e0f-93367aae1611 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:19:07.847907 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aa2c57-25bc-49b2-9e0f-93367aae1611 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:19:07.849405 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aa2c57-25bc-49b2-9e0f-93367aae1611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:19:07.889446 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-90aa2c57-25bc-49b2-9e0f-93367aae1611 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:19:07.891714 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-90aa2c57-25bc-49b2-9e0f-93367aae1611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:19:07.893153 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adc868b-0c9d-4bd8-bbfa-ac88902a6f21 None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:19:07.893662 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adc868b-0c9d-4bd8-bbfa-ac88902a6f21 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:19:07.894162 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adc868b-0c9d-4bd8-bbfa-ac88902a6f21 None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:19:07.897548 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1086/2148] 10.222.0.20 () {54 vars in 921 bytes} [Tue Sep 6 20:19:07 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 20:19:07.907595 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec14d25-a2e6-44d7-a951-4477e3491fa6 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:19:07.908834 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec14d25-a2e6-44d7-a951-4477e3491fa6 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:19:07.909060 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec14d25-a2e6-44d7-a951-4477e3491fa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:19:07.950520 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-5ec14d25-a2e6-44d7-a951-4477e3491fa6 None None] MFA Rules not processed for user `5776cbf4f6714b2083c3ebdd6d98cc87`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:19:07.953664 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-5ec14d25-a2e6-44d7-a951-4477e3491fa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:19:07.990200 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1063/2149] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:19:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:19:08.006951 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aa2c57-25bc-49b2-9e0f-93367aae1611 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:19:08.011820 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-90aa2c57-25bc-49b2-9e0f-93367aae1611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:19:08.147689 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1087/2150] 10.222.0.20 () {58 vars in 1018 bytes} [Tue Sep 6 20:19:07 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:19:08.165361 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec14d25-a2e6-44d7-a951-4477e3491fa6 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:19:08.167299 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-5ec14d25-a2e6-44d7-a951-4477e3491fa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:19:08.179029 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aa2c57-25bc-49b2-9e0f-93367aae1611 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:19:08.191911 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec14d25-a2e6-44d7-a951-4477e3491fa6 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:19:08.197658 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e06c5fd-63fd-4df2-8f88-438f575f81f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:19:08.205382 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e06c5fd-63fd-4df2-8f88-438f575f81f5 service neutron] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:19:08.205382 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e06c5fd-63fd-4df2-8f88-438f575f81f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:19:08.205382 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e06c5fd-63fd-4df2-8f88-438f575f81f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:19:08.208913 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13401fdc-cb45-406a-a9a1-389d8744edf2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:19:08.210369 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13401fdc-cb45-406a-a9a1-389d8744edf2 service neutron] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:19:08.211439 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13401fdc-cb45-406a-a9a1-389d8744edf2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:19:08.211984 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13401fdc-cb45-406a-a9a1-389d8744edf2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:19:08.230457 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e06c5fd-63fd-4df2-8f88-438f575f81f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:19:08.230950 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13401fdc-cb45-406a-a9a1-389d8744edf2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:19:08.233066 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13401fdc-cb45-406a-a9a1-389d8744edf2 service neutron] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:19:08.233778 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e06c5fd-63fd-4df2-8f88-438f575f81f5 service neutron] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:19:08.255385 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1064/2151] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:19:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:19:08.257178 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1088/2152] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:19:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 20:19:08.852290 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e06c5fd-63fd-4df2-8f88-438f575f81f5 service neutron] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:19:08.854444 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-8e06c5fd-63fd-4df2-8f88-438f575f81f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:19:08.878802 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e06c5fd-63fd-4df2-8f88-438f575f81f5 service neutron] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:19:08.905333 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c185b1-4410-4551-8fca-707a27ade4d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:19:08.907971 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c185b1-4410-4551-8fca-707a27ade4d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:19:08.908826 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c185b1-4410-4551-8fca-707a27ade4d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:19:08.909517 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c185b1-4410-4551-8fca-707a27ade4d3 admin admin] PATH_INFO: `/v3/projects/58fce450291d42d2a662047f5d5ffea9` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:19:08.913571 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c185b1-4410-4551-8fca-707a27ade4d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=58fce450291d42d2a662047f5d5ffea9)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:19:08.918012 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c185b1-4410-4551-8fca-707a27ade4d3 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:19:08.995322 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-88c185b1-4410-4551-8fca-707a27ade4d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58fce450291d42d2a662047f5d5ffea9', 'request_id': 'req-88c185b1-4410-4551-8fca-707a27ade4d3'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:19:09.006411 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-88c185b1-4410-4551-8fca-707a27ade4d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58fce450291d42d2a662047f5d5ffea9', 'request_id': 'req-88c185b1-4410-4551-8fca-707a27ade4d3'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:19:09.014481 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:19:09.020389 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-88c185b1-4410-4551-8fca-707a27ade4d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58fce450291d42d2a662047f5d5ffea9', 'request_id': 'req-88c185b1-4410-4551-8fca-707a27ade4d3'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:19:09.045749 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-88c185b1-4410-4551-8fca-707a27ade4d3 admin admin] The token cache is being invalidate because project 58fce450291d42d2a662047f5d5ffea9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81771) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:19:09.046150 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-88c185b1-4410-4551-8fca-707a27ade4d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:19:09.047068 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-88c185b1-4410-4551-8fca-707a27ade4d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:19:09.049110 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1065/2153] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:19:08 2022] DELETE /identity/v3/projects/58fce450291d42d2a662047f5d5ffea9 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:19:09.054278 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13401fdc-cb45-406a-a9a1-389d8744edf2 service neutron] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:19:09.076704 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13401fdc-cb45-406a-a9a1-389d8744edf2 service neutron] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:19:09.197456 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1adc5f-f8ce-42c1-9010-692ef233673b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:19:09.198408 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1adc5f-f8ce-42c1-9010-692ef233673b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:19:09.198889 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1adc5f-f8ce-42c1-9010-692ef233673b admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:19:09.199364 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1adc5f-f8ce-42c1-9010-692ef233673b admin admin] PATH_INFO: `/v3/users/00d9ad180cee4ce997a5f325ac3cea8a` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:19:09.200788 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1adc5f-f8ce-42c1-9010-692ef233673b admin admin] RBAC: Authorizing `identity:delete_user(user_id=00d9ad180cee4ce997a5f325ac3cea8a)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:19:09.203071 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1adc5f-f8ce-42c1-9010-692ef233673b admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:19:09.235317 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:19:09.269929 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-1b1adc5f-f8ce-42c1-9010-692ef233673b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00d9ad180cee4ce997a5f325ac3cea8a'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:19:09.270603 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-1b1adc5f-f8ce-42c1-9010-692ef233673b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00d9ad180cee4ce997a5f325ac3cea8a'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:19:09.289099 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-1b1adc5f-f8ce-42c1-9010-692ef233673b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00d9ad180cee4ce997a5f325ac3cea8a'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:19:09.290000 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-1b1adc5f-f8ce-42c1-9010-692ef233673b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00d9ad180cee4ce997a5f325ac3cea8a'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:19:09.303049 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-1b1adc5f-f8ce-42c1-9010-692ef233673b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00d9ad180cee4ce997a5f325ac3cea8a'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:19:09.322806 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1089/2154] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:19:09 2022] DELETE /identity/v3/users/00d9ad180cee4ce997a5f325ac3cea8a => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:19:09.951892 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c185b1-4410-4551-8fca-707a27ade4d3 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:19:09.955528 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-88c185b1-4410-4551-8fca-707a27ade4d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:19:09.985822 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c185b1-4410-4551-8fca-707a27ade4d3 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:19:10.070636 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269b184b-a289-472b-a9cb-163d88b2193d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:19:10.071481 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269b184b-a289-472b-a9cb-163d88b2193d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:19:10.071781 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269b184b-a289-472b-a9cb-163d88b2193d admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:19:10.072105 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269b184b-a289-472b-a9cb-163d88b2193d admin admin] PATH_INFO: `/v3/projects/42841ec83d874b6e84c77b6de1b9a3c8` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:19:10.073491 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269b184b-a289-472b-a9cb-163d88b2193d admin admin] RBAC: Authorizing `identity:delete_project(project_id=42841ec83d874b6e84c77b6de1b9a3c8)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:19:10.075535 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269b184b-a289-472b-a9cb-163d88b2193d admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:19:10.138567 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-269b184b-a289-472b-a9cb-163d88b2193d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42841ec83d874b6e84c77b6de1b9a3c8', 'request_id': 'req-269b184b-a289-472b-a9cb-163d88b2193d'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:19:10.147064 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-269b184b-a289-472b-a9cb-163d88b2193d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42841ec83d874b6e84c77b6de1b9a3c8', 'request_id': 'req-269b184b-a289-472b-a9cb-163d88b2193d'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:19:10.155708 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-269b184b-a289-472b-a9cb-163d88b2193d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42841ec83d874b6e84c77b6de1b9a3c8', 'request_id': 'req-269b184b-a289-472b-a9cb-163d88b2193d'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:19:10.168150 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-269b184b-a289-472b-a9cb-163d88b2193d admin admin] The token cache is being invalidate because project 42841ec83d874b6e84c77b6de1b9a3c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81771) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:19:10.168432 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-269b184b-a289-472b-a9cb-163d88b2193d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:19:10.169161 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-269b184b-a289-472b-a9cb-163d88b2193d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:19:10.170815 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1066/2155] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:19:09 2022] DELETE /identity/v3/projects/42841ec83d874b6e84c77b6de1b9a3c8 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:19:10.178718 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba46cd2-b912-406d-9a88-c81228e0e000 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:19:10.179091 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba46cd2-b912-406d-9a88-c81228e0e000 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:19:10.179411 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba46cd2-b912-406d-9a88-c81228e0e000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:19:10.202128 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-bba46cd2-b912-406d-9a88-c81228e0e000 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:19:10.204484 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-bba46cd2-b912-406d-9a88-c81228e0e000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:19:10.282928 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1090/2156] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:19:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:19:10.292122 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4704d9f0-659a-4b3e-b342-24c059ed8899 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:19:10.292522 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4704d9f0-659a-4b3e-b342-24c059ed8899 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:19:10.292702 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4704d9f0-659a-4b3e-b342-24c059ed8899 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:19:10.322562 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-4704d9f0-659a-4b3e-b342-24c059ed8899 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:19:10.324363 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-4704d9f0-659a-4b3e-b342-24c059ed8899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:19:10.334069 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1067/2157] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:19:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:19:10.339207 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba46cd2-b912-406d-9a88-c81228e0e000 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:19:10.350403 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba46cd2-b912-406d-9a88-c81228e0e000 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:19:10.355841 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3226d7-a2e3-43b1-a369-0dff9dca05d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:19:10.356513 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3226d7-a2e3-43b1-a369-0dff9dca05d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:19:10.356819 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3226d7-a2e3-43b1-a369-0dff9dca05d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:19:10.357114 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3226d7-a2e3-43b1-a369-0dff9dca05d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:19:10.357885 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3226d7-a2e3-43b1-a369-0dff9dca05d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:19:10.358199 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3226d7-a2e3-43b1-a369-0dff9dca05d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:19:10.360124 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3226d7-a2e3-43b1-a369-0dff9dca05d4 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:19:10.376019 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1091/2158] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 6 20:19:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:19:10.384060 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4704d9f0-659a-4b3e-b342-24c059ed8899 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:19:10.394564 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4704d9f0-659a-4b3e-b342-24c059ed8899 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:19:10.400161 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63cc5ee-314f-4d91-8156-80529af548cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:19:10.400902 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63cc5ee-314f-4d91-8156-80529af548cf admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:19:10.401282 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63cc5ee-314f-4d91-8156-80529af548cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:19:10.401934 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63cc5ee-314f-4d91-8156-80529af548cf admin admin] PATH_INFO: `/v3/projects` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:19:10.402687 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63cc5ee-314f-4d91-8156-80529af548cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:19:10.404719 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63cc5ee-314f-4d91-8156-80529af548cf admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:19:10.440993 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1068/2159] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:19:10 2022] POST /identity/v3/projects => generated 375 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:19:10.445766 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3226d7-a2e3-43b1-a369-0dff9dca05d4 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:19:10.458291 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3226d7-a2e3-43b1-a369-0dff9dca05d4 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:19:10.600833 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d429f45-f03a-44a4-8b94-ee5b1f872553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:19:10.604761 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d429f45-f03a-44a4-8b94-ee5b1f872553 admin admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:19:10.605742 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d429f45-f03a-44a4-8b94-ee5b1f872553 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:19:10.606480 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d429f45-f03a-44a4-8b94-ee5b1f872553 admin admin] PATH_INFO: `/v3/users` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:19:10.607926 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d429f45-f03a-44a4-8b94-ee5b1f872553 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:19:10.614119 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d429f45-f03a-44a4-8b94-ee5b1f872553 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:19:10.660780 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1092/2160] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:19:10 2022] POST /identity/v3/users => generated 288 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:19:10.666113 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63cc5ee-314f-4d91-8156-80529af548cf admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:19:10.679400 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63cc5ee-314f-4d91-8156-80529af548cf admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:19:10.685551 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af64daa4-bbc7-4b0b-90f2-8039e7a77800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:19:10.686502 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af64daa4-bbc7-4b0b-90f2-8039e7a77800 admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:19:10.687702 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af64daa4-bbc7-4b0b-90f2-8039e7a77800 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:19:10.688099 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af64daa4-bbc7-4b0b-90f2-8039e7a77800 admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:19:10.689555 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af64daa4-bbc7-4b0b-90f2-8039e7a77800 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:19:10.691966 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af64daa4-bbc7-4b0b-90f2-8039e7a77800 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:19:10.705318 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1069/2161] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:19:10 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:19:10.711260 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d429f45-f03a-44a4-8b94-ee5b1f872553 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:19:10.739758 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d429f45-f03a-44a4-8b94-ee5b1f872553 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:19:10.749192 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b0ceb0-1385-40f1-996a-adf833d68673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:19:10.750470 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b0ceb0-1385-40f1-996a-adf833d68673 admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:19:10.751008 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b0ceb0-1385-40f1-996a-adf833d68673 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:19:10.751597 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b0ceb0-1385-40f1-996a-adf833d68673 admin admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:19:10.753118 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b0ceb0-1385-40f1-996a-adf833d68673 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:19:10.757128 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b0ceb0-1385-40f1-996a-adf833d68673 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:19:10.800978 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1093/2162] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:19:10 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:19:10.802176 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af64daa4-bbc7-4b0b-90f2-8039e7a77800 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:19:10.819591 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af64daa4-bbc7-4b0b-90f2-8039e7a77800 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:19:10.825043 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b341be30-6ca7-4c35-9c8a-b55e24dbc4ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:19:10.825880 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b341be30-6ca7-4c35-9c8a-b55e24dbc4ca admin admin] REQUEST_METHOD: `PUT` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:19:10.826234 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b341be30-6ca7-4c35-9c8a-b55e24dbc4ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:19:10.826585 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b341be30-6ca7-4c35-9c8a-b55e24dbc4ca admin admin] PATH_INFO: `/v3/projects/7135c6379c864538a0a567d9f30e96f6/users/1c6a010a2fae4bc09daf4c70012b2269/roles/4eea39e24b06431aa357053fe7907a3d` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:19:10.846178 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b341be30-6ca7-4c35-9c8a-b55e24dbc4ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=7135c6379c864538a0a567d9f30e96f6, user_id=1c6a010a2fae4bc09daf4c70012b2269, role_id=4eea39e24b06431aa357053fe7907a3d)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:19:10.849440 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b341be30-6ca7-4c35-9c8a-b55e24dbc4ca admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:19:10.869233 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1070/2163] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:19:10 2022] PUT /identity/v3/projects/7135c6379c864538a0a567d9f30e96f6/users/1c6a010a2fae4bc09daf4c70012b2269/roles/4eea39e24b06431aa357053fe7907a3d => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 20:19:10.881076 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d844c3-f5ec-46d1-8e1d-a56097f5530a None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:19:10.881586 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d844c3-f5ec-46d1-8e1d-a56097f5530a None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:19:10.881978 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d844c3-f5ec-46d1-8e1d-a56097f5530a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:19:10.891943 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:19:10.920307 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-c2d844c3-f5ec-46d1-8e1d-a56097f5530a None None] MFA Rules not processed for user `1c6a010a2fae4bc09daf4c70012b2269`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:19:10.922637 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-c2d844c3-f5ec-46d1-8e1d-a56097f5530a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:19:11.080623 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1094/2164] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:19:10 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:19:11.097352 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b341be30-6ca7-4c35-9c8a-b55e24dbc4ca admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:19:11.099587 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-b341be30-6ca7-4c35-9c8a-b55e24dbc4ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:19:11.124066 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:19:11.217412 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b341be30-6ca7-4c35-9c8a-b55e24dbc4ca admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:19:11.225389 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78e6016-7f1f-4249-a1b5-5233fc24d600 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df1bca993bc142908cc7cf85eabd2b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:19:11.226886 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78e6016-7f1f-4249-a1b5-5233fc24d600 service cinder] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:19:11.227195 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78e6016-7f1f-4249-a1b5-5233fc24d600 service cinder] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:19:11.227480 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78e6016-7f1f-4249-a1b5-5233fc24d600 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:19:11.239694 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78e6016-7f1f-4249-a1b5-5233fc24d600 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:19:11.241283 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78e6016-7f1f-4249-a1b5-5233fc24d600 service cinder] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:19:11.268322 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1071/2165] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:19:11 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:19:14.058709 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bae394c-eb92-44e6-9a2e-c654ea6f9699 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:19:14.059355 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bae394c-eb92-44e6-9a2e-c654ea6f9699 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:19:14.059355 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bae394c-eb92-44e6-9a2e-c654ea6f9699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:19:14.105894 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-2bae394c-eb92-44e6-9a2e-c654ea6f9699 None None] MFA Rules not processed for user `1a52117df70447dba9bbee35d083e44f`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:19:14.108197 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-2bae394c-eb92-44e6-9a2e-c654ea6f9699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:19:14.212050 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1095/2166] 10.222.0.20 () {58 vars in 1009 bytes} [Tue Sep 6 20:19:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:19:14.230811 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78e6016-7f1f-4249-a1b5-5233fc24d600 service cinder] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:19:14.232769 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-d78e6016-7f1f-4249-a1b5-5233fc24d600 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:19:14.362041 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78e6016-7f1f-4249-a1b5-5233fc24d600 service cinder] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:19:14.373474 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6133342-b0eb-4b67-91d3-8b74d5fc86d9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b72516236da47e292d78de201b0552b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:19:14.389559 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6133342-b0eb-4b67-91d3-8b74d5fc86d9 service swift] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:19:14.389559 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6133342-b0eb-4b67-91d3-8b74d5fc86d9 service swift] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:19:14.389559 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6133342-b0eb-4b67-91d3-8b74d5fc86d9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:19:14.401445 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6133342-b0eb-4b67-91d3-8b74d5fc86d9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:19:14.405351 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6133342-b0eb-4b67-91d3-8b74d5fc86d9 service swift] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:19:14.423246 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1072/2167] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Sep 6 20:19:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 06 20:19:15.190454 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bae394c-eb92-44e6-9a2e-c654ea6f9699 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:19:15.352050 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bae394c-eb92-44e6-9a2e-c654ea6f9699 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:19:15.361213 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f553491-b963-41cc-8e66-194590c4a8d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:19:15.362443 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f553491-b963-41cc-8e66-194590c4a8d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:19:15.362975 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f553491-b963-41cc-8e66-194590c4a8d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:19:15.363499 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f553491-b963-41cc-8e66-194590c4a8d5 admin admin] PATH_INFO: `/v3/users/1c6a010a2fae4bc09daf4c70012b2269` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:19:15.365973 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f553491-b963-41cc-8e66-194590c4a8d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c6a010a2fae4bc09daf4c70012b2269)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:19:15.370016 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f553491-b963-41cc-8e66-194590c4a8d5 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:19:15.471342 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:19:15.486448 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-2f553491-b963-41cc-8e66-194590c4a8d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c6a010a2fae4bc09daf4c70012b2269'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:19:15.487603 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-2f553491-b963-41cc-8e66-194590c4a8d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c6a010a2fae4bc09daf4c70012b2269'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:19:15.549526 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-2f553491-b963-41cc-8e66-194590c4a8d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c6a010a2fae4bc09daf4c70012b2269'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:19:15.550678 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-2f553491-b963-41cc-8e66-194590c4a8d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c6a010a2fae4bc09daf4c70012b2269'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:19:15.571201 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-2f553491-b963-41cc-8e66-194590c4a8d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c6a010a2fae4bc09daf4c70012b2269'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:19:15.594716 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1096/2168] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:19:15 2022] DELETE /identity/v3/users/1c6a010a2fae4bc09daf4c70012b2269 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:19:15.598209 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09427be5-205e-45cf-83ee-350cf7ea6aee None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:19:15.598776 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09427be5-205e-45cf-83ee-350cf7ea6aee None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:19:15.599337 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09427be5-205e-45cf-83ee-350cf7ea6aee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:19:15.622805 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-09427be5-205e-45cf-83ee-350cf7ea6aee None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:19:15.624205 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-09427be5-205e-45cf-83ee-350cf7ea6aee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:19:15.723133 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1073/2169] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:19:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:19:15.737403 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f553491-b963-41cc-8e66-194590c4a8d5 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:19:15.739787 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-2f553491-b963-41cc-8e66-194590c4a8d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:19:15.870781 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f553491-b963-41cc-8e66-194590c4a8d5 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:19:15.881562 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8578f6c-15c7-483d-b6f5-ecf101e15f38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:19:15.882694 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8578f6c-15c7-483d-b6f5-ecf101e15f38 service neutron] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:19:15.883206 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8578f6c-15c7-483d-b6f5-ecf101e15f38 service neutron] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:19:15.883713 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8578f6c-15c7-483d-b6f5-ecf101e15f38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:19:15.897961 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8578f6c-15c7-483d-b6f5-ecf101e15f38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:19:15.899776 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8578f6c-15c7-483d-b6f5-ecf101e15f38 service neutron] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:19:15.920248 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1097/2170] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:19:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:19:16.589990 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09427be5-205e-45cf-83ee-350cf7ea6aee None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:19:16.591787 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-09427be5-205e-45cf-83ee-350cf7ea6aee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:19:16.607912 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09427be5-205e-45cf-83ee-350cf7ea6aee None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:19:16.613752 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3234201-aeb4-4db3-af9b-18ccb513fbf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:19:16.614429 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3234201-aeb4-4db3-af9b-18ccb513fbf4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:19:16.614772 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3234201-aeb4-4db3-af9b-18ccb513fbf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:19:16.615045 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3234201-aeb4-4db3-af9b-18ccb513fbf4 admin admin] PATH_INFO: `/v3/projects/7135c6379c864538a0a567d9f30e96f6` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:19:16.617195 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3234201-aeb4-4db3-af9b-18ccb513fbf4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7135c6379c864538a0a567d9f30e96f6)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:19:16.634707 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3234201-aeb4-4db3-af9b-18ccb513fbf4 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:19:16.800986 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:19:16.814188 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-d3234201-aeb4-4db3-af9b-18ccb513fbf4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7135c6379c864538a0a567d9f30e96f6', 'request_id': 'req-d3234201-aeb4-4db3-af9b-18ccb513fbf4'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:19:16.828733 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-d3234201-aeb4-4db3-af9b-18ccb513fbf4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7135c6379c864538a0a567d9f30e96f6', 'request_id': 'req-d3234201-aeb4-4db3-af9b-18ccb513fbf4'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:19:16.843363 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-d3234201-aeb4-4db3-af9b-18ccb513fbf4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7135c6379c864538a0a567d9f30e96f6', 'request_id': 'req-d3234201-aeb4-4db3-af9b-18ccb513fbf4'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:19:16.874378 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-d3234201-aeb4-4db3-af9b-18ccb513fbf4 admin admin] The token cache is being invalidate because project 7135c6379c864538a0a567d9f30e96f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81771) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:19:16.875067 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-d3234201-aeb4-4db3-af9b-18ccb513fbf4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:19:16.876303 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-d3234201-aeb4-4db3-af9b-18ccb513fbf4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:19:16.883236 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1074/2171] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:19:16 2022] DELETE /identity/v3/projects/7135c6379c864538a0a567d9f30e96f6 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 20:19:16.891633 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82729f01-581f-4b38-9eac-c8f4e492411f None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:19:16.892623 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82729f01-581f-4b38-9eac-c8f4e492411f None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:19:16.892623 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82729f01-581f-4b38-9eac-c8f4e492411f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:19:16.988711 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-82729f01-581f-4b38-9eac-c8f4e492411f None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:19:16.988711 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-82729f01-581f-4b38-9eac-c8f4e492411f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:19:17.179986 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1098/2172] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:19:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:19:17.182654 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa89511-376f-4d64-93a0-f7516de305c3 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:19:17.183196 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa89511-376f-4d64-93a0-f7516de305c3 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:19:17.183626 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa89511-376f-4d64-93a0-f7516de305c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:19:17.215634 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-9aa89511-376f-4d64-93a0-f7516de305c3 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:19:17.217402 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-9aa89511-376f-4d64-93a0-f7516de305c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:19:17.234265 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1075/2173] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:19:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:19:17.241275 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82729f01-581f-4b38-9eac-c8f4e492411f None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:19:17.266038 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82729f01-581f-4b38-9eac-c8f4e492411f None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:19:17.283737 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d9d4aa-4432-4d67-86bd-66fb05b9658c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:19:17.284977 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d9d4aa-4432-4d67-86bd-66fb05b9658c admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:19:17.285460 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d9d4aa-4432-4d67-86bd-66fb05b9658c admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:19:17.286056 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d9d4aa-4432-4d67-86bd-66fb05b9658c admin admin] PATH_INFO: `/v3/domains` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:19:17.287251 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d9d4aa-4432-4d67-86bd-66fb05b9658c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:19:17.287807 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d9d4aa-4432-4d67-86bd-66fb05b9658c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:19:17.291334 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d9d4aa-4432-4d67-86bd-66fb05b9658c admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:19:17.322693 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1099/2174] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 6 20:19:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:19:17.324198 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa89511-376f-4d64-93a0-f7516de305c3 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:19:17.365479 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa89511-376f-4d64-93a0-f7516de305c3 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:19:17.387784 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f114643-5e12-41f1-a31b-f17fcef1d2c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:19:17.389394 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f114643-5e12-41f1-a31b-f17fcef1d2c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:19:17.390224 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f114643-5e12-41f1-a31b-f17fcef1d2c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:19:17.391043 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f114643-5e12-41f1-a31b-f17fcef1d2c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:19:17.392687 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f114643-5e12-41f1-a31b-f17fcef1d2c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:19:17.402635 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f114643-5e12-41f1-a31b-f17fcef1d2c5 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:19:17.480063 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1076/2175] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:19:17 2022] POST /identity/v3/projects => generated 397 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:19:17.489518 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d9d4aa-4432-4d67-86bd-66fb05b9658c admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:19:17.508652 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d9d4aa-4432-4d67-86bd-66fb05b9658c admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:19:17.643842 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:19:17.741874 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db27a153-fb27-453a-85a4-e1003595b2e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:19:17.741874 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db27a153-fb27-453a-85a4-e1003595b2e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:19:17.741874 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db27a153-fb27-453a-85a4-e1003595b2e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:19:17.741874 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db27a153-fb27-453a-85a4-e1003595b2e8 admin admin] PATH_INFO: `/v3/users` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:19:17.742780 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db27a153-fb27-453a-85a4-e1003595b2e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:19:17.746952 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db27a153-fb27-453a-85a4-e1003595b2e8 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:19:17.791861 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1100/2176] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:19:17 2022] POST /identity/v3/users => generated 299 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:19:17.798978 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f114643-5e12-41f1-a31b-f17fcef1d2c5 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:19:17.823118 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f114643-5e12-41f1-a31b-f17fcef1d2c5 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:19:17.846307 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc83dadd-649a-44ef-99dc-597661bcdb5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:19:17.847623 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc83dadd-649a-44ef-99dc-597661bcdb5a admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:19:17.848384 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc83dadd-649a-44ef-99dc-597661bcdb5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:19:17.849153 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc83dadd-649a-44ef-99dc-597661bcdb5a admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:19:17.851097 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc83dadd-649a-44ef-99dc-597661bcdb5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:19:17.855967 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc83dadd-649a-44ef-99dc-597661bcdb5a admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:19:17.883916 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1077/2177] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:19:17 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:19:17.891033 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db27a153-fb27-453a-85a4-e1003595b2e8 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:19:17.919083 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db27a153-fb27-453a-85a4-e1003595b2e8 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:19:17.930321 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d68c17d-7999-4620-9c3b-2ea376a3102a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:19:17.932520 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d68c17d-7999-4620-9c3b-2ea376a3102a admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:19:17.933313 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d68c17d-7999-4620-9c3b-2ea376a3102a admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:19:17.933925 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d68c17d-7999-4620-9c3b-2ea376a3102a admin admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:19:17.935612 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d68c17d-7999-4620-9c3b-2ea376a3102a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:19:17.940712 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d68c17d-7999-4620-9c3b-2ea376a3102a admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:19:17.974562 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1101/2178] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:19:17 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:19:17.975198 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc83dadd-649a-44ef-99dc-597661bcdb5a admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:19:17.998582 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc83dadd-649a-44ef-99dc-597661bcdb5a admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:19:18.009028 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804c8698-1482-47d8-8846-877d7567c86a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:19:18.010562 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804c8698-1482-47d8-8846-877d7567c86a admin admin] REQUEST_METHOD: `PUT` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:19:18.011234 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804c8698-1482-47d8-8846-877d7567c86a admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:19:18.011883 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804c8698-1482-47d8-8846-877d7567c86a admin admin] PATH_INFO: `/v3/projects/531b307a59304c9f91be2423440681a9/users/030a7ade74424c53a2a3c60a8de1c43c/roles/4eea39e24b06431aa357053fe7907a3d` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:19:18.055418 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804c8698-1482-47d8-8846-877d7567c86a admin admin] RBAC: Authorizing `identity:create_grant(project_id=531b307a59304c9f91be2423440681a9, user_id=030a7ade74424c53a2a3c60a8de1c43c, role_id=4eea39e24b06431aa357053fe7907a3d)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:19:18.058376 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804c8698-1482-47d8-8846-877d7567c86a admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:19:18.105050 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1078/2179] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:19:17 2022] PUT /identity/v3/projects/531b307a59304c9f91be2423440681a9/users/030a7ade74424c53a2a3c60a8de1c43c/roles/4eea39e24b06431aa357053fe7907a3d => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 20:19:18.110887 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f9dfcb-99b2-46d7-96ab-d5e580feba19 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:19:18.111316 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f9dfcb-99b2-46d7-96ab-d5e580feba19 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:19:18.111917 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f9dfcb-99b2-46d7-96ab-d5e580feba19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:19:18.211696 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-38f9dfcb-99b2-46d7-96ab-d5e580feba19 None None] MFA Rules not processed for user `030a7ade74424c53a2a3c60a8de1c43c`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:19:18.214723 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-38f9dfcb-99b2-46d7-96ab-d5e580feba19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:19:18.379167 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1102/2180] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:19:18 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:19:18.458489 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804c8698-1482-47d8-8846-877d7567c86a admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:19:18.470868 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-804c8698-1482-47d8-8846-877d7567c86a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:19:18.698681 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804c8698-1482-47d8-8846-877d7567c86a admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:19:18.707460 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ad5c38-5b1b-44e4-bcd8-b7470a918fa6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df1bca993bc142908cc7cf85eabd2b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:19:18.708826 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ad5c38-5b1b-44e4-bcd8-b7470a918fa6 service cinder] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:19:18.709313 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ad5c38-5b1b-44e4-bcd8-b7470a918fa6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:19:18.710494 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ad5c38-5b1b-44e4-bcd8-b7470a918fa6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:19:18.733335 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ad5c38-5b1b-44e4-bcd8-b7470a918fa6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:19:18.736891 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ad5c38-5b1b-44e4-bcd8-b7470a918fa6 service cinder] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:19:18.790926 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1079/2181] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:19:18 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:19:28.364141 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=44,keystone:INSERT=3 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:19:28.753738 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=62,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:19:34.803955 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f9dfcb-99b2-46d7-96ab-d5e580feba19 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:19:34.921364 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f9dfcb-99b2-46d7-96ab-d5e580feba19 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:19:34.930737 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a230ef-76ab-4d06-aefe-9b05bb06d787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:19:34.932271 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a230ef-76ab-4d06-aefe-9b05bb06d787 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:19:34.933075 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a230ef-76ab-4d06-aefe-9b05bb06d787 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:19:34.934536 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a230ef-76ab-4d06-aefe-9b05bb06d787 admin admin] PATH_INFO: `/v3/users/030a7ade74424c53a2a3c60a8de1c43c` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:19:34.937583 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a230ef-76ab-4d06-aefe-9b05bb06d787 admin admin] RBAC: Authorizing `identity:delete_user(user_id=030a7ade74424c53a2a3c60a8de1c43c)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:19:34.941718 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a230ef-76ab-4d06-aefe-9b05bb06d787 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:19:35.048070 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-63a230ef-76ab-4d06-aefe-9b05bb06d787 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '030a7ade74424c53a2a3c60a8de1c43c'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:19:35.049116 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-63a230ef-76ab-4d06-aefe-9b05bb06d787 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '030a7ade74424c53a2a3c60a8de1c43c'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:19:35.076153 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-63a230ef-76ab-4d06-aefe-9b05bb06d787 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '030a7ade74424c53a2a3c60a8de1c43c'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:19:35.077908 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-63a230ef-76ab-4d06-aefe-9b05bb06d787 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '030a7ade74424c53a2a3c60a8de1c43c'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:19:35.113476 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-63a230ef-76ab-4d06-aefe-9b05bb06d787 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '030a7ade74424c53a2a3c60a8de1c43c'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:19:35.144749 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1103/2182] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:19:34 2022] DELETE /identity/v3/users/030a7ade74424c53a2a3c60a8de1c43c => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:19:35.151436 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8740d694-1072-4876-8ea6-18b5d313ff65 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:19:35.152335 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8740d694-1072-4876-8ea6-18b5d313ff65 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:19:35.152616 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8740d694-1072-4876-8ea6-18b5d313ff65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:19:35.198883 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-8740d694-1072-4876-8ea6-18b5d313ff65 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:19:35.201703 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-8740d694-1072-4876-8ea6-18b5d313ff65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:19:35.342891 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1080/2183] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:19:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:19:35.357470 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a230ef-76ab-4d06-aefe-9b05bb06d787 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:19:35.361535 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-63a230ef-76ab-4d06-aefe-9b05bb06d787 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:19:35.505016 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a230ef-76ab-4d06-aefe-9b05bb06d787 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:19:35.509766 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b744e67-6928-46f7-9a1b-d0d2ee90339e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:19:35.510758 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b744e67-6928-46f7-9a1b-d0d2ee90339e service neutron] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:19:35.511040 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b744e67-6928-46f7-9a1b-d0d2ee90339e service neutron] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:19:35.511349 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b744e67-6928-46f7-9a1b-d0d2ee90339e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:19:35.528934 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b744e67-6928-46f7-9a1b-d0d2ee90339e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:19:35.531150 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b744e67-6928-46f7-9a1b-d0d2ee90339e service neutron] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:19:35.558629 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1104/2184] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:19:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 20:19:36.179913 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8740d694-1072-4876-8ea6-18b5d313ff65 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:19:36.182287 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-8740d694-1072-4876-8ea6-18b5d313ff65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:19:36.201917 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8740d694-1072-4876-8ea6-18b5d313ff65 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:19:36.227265 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1248ad-c23c-4da5-b2da-6dc9fae45d88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:19:36.229728 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1248ad-c23c-4da5-b2da-6dc9fae45d88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:19:36.230564 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1248ad-c23c-4da5-b2da-6dc9fae45d88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:19:36.232215 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1248ad-c23c-4da5-b2da-6dc9fae45d88 admin admin] PATH_INFO: `/v3/projects/531b307a59304c9f91be2423440681a9` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:19:36.235868 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1248ad-c23c-4da5-b2da-6dc9fae45d88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=531b307a59304c9f91be2423440681a9)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:19:36.246241 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1248ad-c23c-4da5-b2da-6dc9fae45d88 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:19:36.361011 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-ee1248ad-c23c-4da5-b2da-6dc9fae45d88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '531b307a59304c9f91be2423440681a9', 'request_id': 'req-ee1248ad-c23c-4da5-b2da-6dc9fae45d88'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:19:36.380630 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-ee1248ad-c23c-4da5-b2da-6dc9fae45d88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '531b307a59304c9f91be2423440681a9', 'request_id': 'req-ee1248ad-c23c-4da5-b2da-6dc9fae45d88'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:19:36.398211 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-ee1248ad-c23c-4da5-b2da-6dc9fae45d88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '531b307a59304c9f91be2423440681a9', 'request_id': 'req-ee1248ad-c23c-4da5-b2da-6dc9fae45d88'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:19:36.417930 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-ee1248ad-c23c-4da5-b2da-6dc9fae45d88 admin admin] The token cache is being invalidate because project 531b307a59304c9f91be2423440681a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81771) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:19:36.418420 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-ee1248ad-c23c-4da5-b2da-6dc9fae45d88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:19:36.422294 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-ee1248ad-c23c-4da5-b2da-6dc9fae45d88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:19:36.425117 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1081/2185] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:19:36 2022] DELETE /identity/v3/projects/531b307a59304c9f91be2423440681a9 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:19:40.073247 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b744e67-6928-46f7-9a1b-d0d2ee90339e service neutron] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:19:40.079642 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-9b744e67-6928-46f7-9a1b-d0d2ee90339e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:19:40.107147 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b744e67-6928-46f7-9a1b-d0d2ee90339e service neutron] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:19:40.148024 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:19:40.215822 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776b129a-aad7-408a-b7c4-c29903def105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:19:40.216699 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776b129a-aad7-408a-b7c4-c29903def105 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:19:40.217123 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776b129a-aad7-408a-b7c4-c29903def105 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:19:40.217555 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776b129a-aad7-408a-b7c4-c29903def105 admin admin] PATH_INFO: `/v3/users/060a287ebe6944af8f2947e6af97703c` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:19:40.219620 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776b129a-aad7-408a-b7c4-c29903def105 admin admin] RBAC: Authorizing `identity:delete_user(user_id=060a287ebe6944af8f2947e6af97703c)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:19:40.222891 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776b129a-aad7-408a-b7c4-c29903def105 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:19:40.310972 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-776b129a-aad7-408a-b7c4-c29903def105 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '060a287ebe6944af8f2947e6af97703c'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:19:40.311938 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-776b129a-aad7-408a-b7c4-c29903def105 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '060a287ebe6944af8f2947e6af97703c'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:19:40.344772 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-776b129a-aad7-408a-b7c4-c29903def105 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '060a287ebe6944af8f2947e6af97703c'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:19:40.346506 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-776b129a-aad7-408a-b7c4-c29903def105 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '060a287ebe6944af8f2947e6af97703c'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:19:40.360288 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-776b129a-aad7-408a-b7c4-c29903def105 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '060a287ebe6944af8f2947e6af97703c'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:19:40.384590 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1105/2186] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:19:40 2022] DELETE /identity/v3/users/060a287ebe6944af8f2947e6af97703c => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:19:40.391694 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be67bced-a87a-4d4e-8ef1-c8c1b89ca117 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:19:40.392356 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be67bced-a87a-4d4e-8ef1-c8c1b89ca117 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:19:40.392635 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be67bced-a87a-4d4e-8ef1-c8c1b89ca117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:19:40.427785 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-be67bced-a87a-4d4e-8ef1-c8c1b89ca117 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:19:40.430017 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-be67bced-a87a-4d4e-8ef1-c8c1b89ca117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:19:40.522655 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1082/2187] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:19:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:19:40.535073 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776b129a-aad7-408a-b7c4-c29903def105 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:19:40.537783 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-776b129a-aad7-408a-b7c4-c29903def105 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:19:40.667080 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776b129a-aad7-408a-b7c4-c29903def105 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:19:40.674282 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd57e56-d117-4bbf-ab6c-8b1427cf5ffc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:19:40.675661 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd57e56-d117-4bbf-ab6c-8b1427cf5ffc service neutron] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:19:40.676204 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd57e56-d117-4bbf-ab6c-8b1427cf5ffc service neutron] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:19:40.676737 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd57e56-d117-4bbf-ab6c-8b1427cf5ffc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:19:40.696171 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd57e56-d117-4bbf-ab6c-8b1427cf5ffc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:19:40.699366 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd57e56-d117-4bbf-ab6c-8b1427cf5ffc service neutron] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:19:40.720414 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1106/2188] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:19:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:19:41.228395 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be67bced-a87a-4d4e-8ef1-c8c1b89ca117 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:19:41.230794 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-be67bced-a87a-4d4e-8ef1-c8c1b89ca117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:19:41.253214 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be67bced-a87a-4d4e-8ef1-c8c1b89ca117 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:19:41.266741 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3ce675-afef-45b7-9615-cecb76e997c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:19:41.268680 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3ce675-afef-45b7-9615-cecb76e997c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:19:41.269573 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3ce675-afef-45b7-9615-cecb76e997c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:19:41.270322 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3ce675-afef-45b7-9615-cecb76e997c4 admin admin] PATH_INFO: `/v3/projects/258bdd28ff27449e8fa3bab0678b6aff` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:19:41.273815 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3ce675-afef-45b7-9615-cecb76e997c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=258bdd28ff27449e8fa3bab0678b6aff)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:19:41.278712 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3ce675-afef-45b7-9615-cecb76e997c4 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:19:41.329399 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:19:41.395741 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-6d3ce675-afef-45b7-9615-cecb76e997c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '258bdd28ff27449e8fa3bab0678b6aff', 'request_id': 'req-6d3ce675-afef-45b7-9615-cecb76e997c4'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:19:41.411240 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-6d3ce675-afef-45b7-9615-cecb76e997c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '258bdd28ff27449e8fa3bab0678b6aff', 'request_id': 'req-6d3ce675-afef-45b7-9615-cecb76e997c4'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:19:41.427055 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-6d3ce675-afef-45b7-9615-cecb76e997c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '258bdd28ff27449e8fa3bab0678b6aff', 'request_id': 'req-6d3ce675-afef-45b7-9615-cecb76e997c4'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:19:41.453965 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-6d3ce675-afef-45b7-9615-cecb76e997c4 admin admin] The token cache is being invalidate because project 258bdd28ff27449e8fa3bab0678b6aff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81771) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:19:41.454618 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-6d3ce675-afef-45b7-9615-cecb76e997c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:19:41.456923 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-6d3ce675-afef-45b7-9615-cecb76e997c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:19:41.465310 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1083/2189] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:19:41 2022] DELETE /identity/v3/projects/258bdd28ff27449e8fa3bab0678b6aff => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:19:41.468165 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136fce21-a9a4-4187-b2d1-261eb98172ae None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:19:41.468998 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136fce21-a9a4-4187-b2d1-261eb98172ae None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:19:41.469731 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136fce21-a9a4-4187-b2d1-261eb98172ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:19:41.507093 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-136fce21-a9a4-4187-b2d1-261eb98172ae None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:19:41.510008 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-136fce21-a9a4-4187-b2d1-261eb98172ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:19:41.611425 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:19:41.692627 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1107/2190] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:19:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:19:41.707130 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfece54-9233-4b9f-ac58-6d8ae17227ad None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:19:41.707600 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfece54-9233-4b9f-ac58-6d8ae17227ad None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:19:41.708376 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfece54-9233-4b9f-ac58-6d8ae17227ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:19:41.744006 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-ebfece54-9233-4b9f-ac58-6d8ae17227ad None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:19:41.746362 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-ebfece54-9233-4b9f-ac58-6d8ae17227ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:19:41.762518 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1084/2191] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:19:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:19:41.763460 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136fce21-a9a4-4187-b2d1-261eb98172ae None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:19:41.796259 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136fce21-a9a4-4187-b2d1-261eb98172ae None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:19:41.809066 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b06bc9-88cf-48cf-8b18-46c0827743eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:19:41.809826 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b06bc9-88cf-48cf-8b18-46c0827743eb admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:19:41.810938 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b06bc9-88cf-48cf-8b18-46c0827743eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:19:41.810938 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b06bc9-88cf-48cf-8b18-46c0827743eb admin admin] PATH_INFO: `/v3/domains` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:19:41.811280 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b06bc9-88cf-48cf-8b18-46c0827743eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:19:41.811417 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b06bc9-88cf-48cf-8b18-46c0827743eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:19:41.814268 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b06bc9-88cf-48cf-8b18-46c0827743eb admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:19:41.831964 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1108/2192] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 6 20:19:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 20:19:41.838519 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfece54-9233-4b9f-ac58-6d8ae17227ad None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:19:41.858167 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfece54-9233-4b9f-ac58-6d8ae17227ad None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:19:41.866764 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0e57e2-a699-41a5-a382-e7008f6d8695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:19:41.867921 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0e57e2-a699-41a5-a382-e7008f6d8695 admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:19:41.868500 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0e57e2-a699-41a5-a382-e7008f6d8695 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:19:41.869077 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0e57e2-a699-41a5-a382-e7008f6d8695 admin admin] PATH_INFO: `/v3/projects` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:19:41.870392 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0e57e2-a699-41a5-a382-e7008f6d8695 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:19:41.874393 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0e57e2-a699-41a5-a382-e7008f6d8695 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:19:41.928341 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1085/2193] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:19:41 2022] POST /identity/v3/projects => generated 383 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:19:41.933304 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b06bc9-88cf-48cf-8b18-46c0827743eb admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:19:41.945723 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b06bc9-88cf-48cf-8b18-46c0827743eb admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:19:42.038382 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be20833d-b94d-4f05-a053-1b14d3ed4480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:19:42.039415 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be20833d-b94d-4f05-a053-1b14d3ed4480 admin admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:19:42.039842 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be20833d-b94d-4f05-a053-1b14d3ed4480 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:19:42.040419 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be20833d-b94d-4f05-a053-1b14d3ed4480 admin admin] PATH_INFO: `/v3/users` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:19:42.042657 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be20833d-b94d-4f05-a053-1b14d3ed4480 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:19:42.046943 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be20833d-b94d-4f05-a053-1b14d3ed4480 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:19:42.076795 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1109/2194] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:19:41 2022] POST /identity/v3/users => generated 292 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:19:42.082753 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0e57e2-a699-41a5-a382-e7008f6d8695 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:19:42.102181 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0e57e2-a699-41a5-a382-e7008f6d8695 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:19:42.110616 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5588edf9-b812-4e02-ad13-e67459c32987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:19:42.111697 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5588edf9-b812-4e02-ad13-e67459c32987 admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:19:42.112393 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5588edf9-b812-4e02-ad13-e67459c32987 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:19:42.113308 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5588edf9-b812-4e02-ad13-e67459c32987 admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:19:42.114588 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5588edf9-b812-4e02-ad13-e67459c32987 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:19:42.117851 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5588edf9-b812-4e02-ad13-e67459c32987 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:19:42.136535 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1086/2195] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:19:42 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:19:42.143370 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be20833d-b94d-4f05-a053-1b14d3ed4480 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:19:42.157503 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be20833d-b94d-4f05-a053-1b14d3ed4480 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:19:42.162820 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d247915-d92c-445a-ab1d-3f71d7b37e25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:19:42.163455 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d247915-d92c-445a-ab1d-3f71d7b37e25 admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:19:42.163799 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d247915-d92c-445a-ab1d-3f71d7b37e25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:19:42.164080 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d247915-d92c-445a-ab1d-3f71d7b37e25 admin admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:19:42.165117 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d247915-d92c-445a-ab1d-3f71d7b37e25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:19:42.166877 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d247915-d92c-445a-ab1d-3f71d7b37e25 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:19:42.181270 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1110/2196] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:19:42 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:19:42.188738 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5588edf9-b812-4e02-ad13-e67459c32987 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:19:42.210641 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5588edf9-b812-4e02-ad13-e67459c32987 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:19:42.219747 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a87bef-5054-4d44-9af7-56ca9a120e90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:19:42.220948 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a87bef-5054-4d44-9af7-56ca9a120e90 admin admin] REQUEST_METHOD: `PUT` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:19:42.221458 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a87bef-5054-4d44-9af7-56ca9a120e90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:19:42.221970 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a87bef-5054-4d44-9af7-56ca9a120e90 admin admin] PATH_INFO: `/v3/projects/b887285389c04d2cb53ccf0820374e54/users/cc6dc323d7dd41bfa9bd2ddacbdaf8fe/roles/4eea39e24b06431aa357053fe7907a3d` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:19:42.264939 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a87bef-5054-4d44-9af7-56ca9a120e90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b887285389c04d2cb53ccf0820374e54, user_id=cc6dc323d7dd41bfa9bd2ddacbdaf8fe, role_id=4eea39e24b06431aa357053fe7907a3d)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:19:42.278948 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a87bef-5054-4d44-9af7-56ca9a120e90 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:19:42.307419 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1087/2197] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:19:42 2022] PUT /identity/v3/projects/b887285389c04d2cb53ccf0820374e54/users/cc6dc323d7dd41bfa9bd2ddacbdaf8fe/roles/4eea39e24b06431aa357053fe7907a3d => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:19:42.335174 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b0a453-7510-4558-95bd-ab552e27fef0 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:19:42.336027 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b0a453-7510-4558-95bd-ab552e27fef0 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:19:42.337728 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b0a453-7510-4558-95bd-ab552e27fef0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:19:42.390111 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-e3b0a453-7510-4558-95bd-ab552e27fef0 None None] MFA Rules not processed for user `cc6dc323d7dd41bfa9bd2ddacbdaf8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:19:42.392258 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-e3b0a453-7510-4558-95bd-ab552e27fef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:19:42.547447 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1111/2198] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:19:42 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:19:42.563459 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a87bef-5054-4d44-9af7-56ca9a120e90 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:19:42.567320 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-e3a87bef-5054-4d44-9af7-56ca9a120e90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:19:42.691186 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a87bef-5054-4d44-9af7-56ca9a120e90 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:19:42.697766 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd83230-c7ce-4cdb-a864-40132d739ef5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df1bca993bc142908cc7cf85eabd2b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:19:42.698848 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd83230-c7ce-4cdb-a864-40132d739ef5 service cinder] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:19:42.699323 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd83230-c7ce-4cdb-a864-40132d739ef5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:19:42.699812 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd83230-c7ce-4cdb-a864-40132d739ef5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:19:42.721202 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd83230-c7ce-4cdb-a864-40132d739ef5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:19:42.724323 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd83230-c7ce-4cdb-a864-40132d739ef5 service cinder] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:19:42.742893 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1088/2199] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:19:42 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:19:45.451347 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301eb9d9-e73e-4976-b29f-dba66463be3b None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:19:45.452152 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301eb9d9-e73e-4976-b29f-dba66463be3b None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:19:45.452498 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301eb9d9-e73e-4976-b29f-dba66463be3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:19:45.475471 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-301eb9d9-e73e-4976-b29f-dba66463be3b None None] MFA Rules not processed for user `1a52117df70447dba9bbee35d083e44f`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:19:45.476812 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-301eb9d9-e73e-4976-b29f-dba66463be3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:19:45.521984 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:19:45.548887 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1112/2200] 10.222.0.20 () {58 vars in 967 bytes} [Tue Sep 6 20:19:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:19:45.577056 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd83230-c7ce-4cdb-a864-40132d739ef5 service cinder] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:19:45.579142 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-edd83230-c7ce-4cdb-a864-40132d739ef5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:19:45.646688 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:19:45.673536 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd83230-c7ce-4cdb-a864-40132d739ef5 service cinder] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:19:45.679055 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04774984-7733-4d77-a231-14247139f43d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b72516236da47e292d78de201b0552b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:19:45.679692 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04774984-7733-4d77-a231-14247139f43d service swift] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:19:45.679979 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04774984-7733-4d77-a231-14247139f43d service swift] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:19:45.680264 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04774984-7733-4d77-a231-14247139f43d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:19:45.691245 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04774984-7733-4d77-a231-14247139f43d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:19:45.693004 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04774984-7733-4d77-a231-14247139f43d service swift] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:19:45.702286 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1089/2201] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Sep 6 20:19:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 20:19:45.737680 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301eb9d9-e73e-4976-b29f-dba66463be3b None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:19:45.856361 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301eb9d9-e73e-4976-b29f-dba66463be3b None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:19:45.865152 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283fc295-eaa0-457d-ab92-0a990b39371b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:19:45.866002 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283fc295-eaa0-457d-ab92-0a990b39371b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:19:45.866490 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283fc295-eaa0-457d-ab92-0a990b39371b admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:19:45.866957 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283fc295-eaa0-457d-ab92-0a990b39371b admin admin] PATH_INFO: `/v3/users/cc6dc323d7dd41bfa9bd2ddacbdaf8fe` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:19:45.870703 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283fc295-eaa0-457d-ab92-0a990b39371b admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc6dc323d7dd41bfa9bd2ddacbdaf8fe)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:19:45.873335 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283fc295-eaa0-457d-ab92-0a990b39371b admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:19:45.967933 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-283fc295-eaa0-457d-ab92-0a990b39371b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc6dc323d7dd41bfa9bd2ddacbdaf8fe'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:19:45.969169 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-283fc295-eaa0-457d-ab92-0a990b39371b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc6dc323d7dd41bfa9bd2ddacbdaf8fe'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:19:46.008786 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-283fc295-eaa0-457d-ab92-0a990b39371b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc6dc323d7dd41bfa9bd2ddacbdaf8fe'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:19:46.010088 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-283fc295-eaa0-457d-ab92-0a990b39371b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc6dc323d7dd41bfa9bd2ddacbdaf8fe'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:19:46.031520 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-283fc295-eaa0-457d-ab92-0a990b39371b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc6dc323d7dd41bfa9bd2ddacbdaf8fe'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:19:46.059904 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1113/2202] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:19:45 2022] DELETE /identity/v3/users/cc6dc323d7dd41bfa9bd2ddacbdaf8fe => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:19:46.067783 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530170fd-b11f-43ea-9cff-66562bb50f37 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:19:46.068116 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530170fd-b11f-43ea-9cff-66562bb50f37 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:19:46.068405 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530170fd-b11f-43ea-9cff-66562bb50f37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:19:46.097257 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-530170fd-b11f-43ea-9cff-66562bb50f37 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:19:46.100611 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-530170fd-b11f-43ea-9cff-66562bb50f37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:19:46.186780 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1090/2203] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:19:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:19:46.197847 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283fc295-eaa0-457d-ab92-0a990b39371b admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:19:46.199435 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-283fc295-eaa0-457d-ab92-0a990b39371b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:19:46.274690 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283fc295-eaa0-457d-ab92-0a990b39371b admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:19:46.279287 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdacebc4-0555-486a-b9a5-7bf9836f9ecc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:19:46.280049 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdacebc4-0555-486a-b9a5-7bf9836f9ecc service neutron] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:19:46.280429 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdacebc4-0555-486a-b9a5-7bf9836f9ecc service neutron] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:19:46.280839 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdacebc4-0555-486a-b9a5-7bf9836f9ecc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:19:46.292565 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdacebc4-0555-486a-b9a5-7bf9836f9ecc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:19:46.294524 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdacebc4-0555-486a-b9a5-7bf9836f9ecc service neutron] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:19:46.306723 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1114/2204] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:19:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:19:46.877650 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530170fd-b11f-43ea-9cff-66562bb50f37 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:19:46.879691 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-530170fd-b11f-43ea-9cff-66562bb50f37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:19:46.899392 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530170fd-b11f-43ea-9cff-66562bb50f37 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:19:46.906671 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c5495b-1d93-472b-8a50-ce80a5f72ff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:19:46.907551 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c5495b-1d93-472b-8a50-ce80a5f72ff6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:19:46.907899 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c5495b-1d93-472b-8a50-ce80a5f72ff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:19:46.908255 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c5495b-1d93-472b-8a50-ce80a5f72ff6 admin admin] PATH_INFO: `/v3/projects/b887285389c04d2cb53ccf0820374e54` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:19:46.909837 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c5495b-1d93-472b-8a50-ce80a5f72ff6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b887285389c04d2cb53ccf0820374e54)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:19:46.912892 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c5495b-1d93-472b-8a50-ce80a5f72ff6 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:19:46.976340 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-71c5495b-1d93-472b-8a50-ce80a5f72ff6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b887285389c04d2cb53ccf0820374e54', 'request_id': 'req-71c5495b-1d93-472b-8a50-ce80a5f72ff6'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:19:46.984136 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-71c5495b-1d93-472b-8a50-ce80a5f72ff6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b887285389c04d2cb53ccf0820374e54', 'request_id': 'req-71c5495b-1d93-472b-8a50-ce80a5f72ff6'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:19:46.994745 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-71c5495b-1d93-472b-8a50-ce80a5f72ff6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b887285389c04d2cb53ccf0820374e54', 'request_id': 'req-71c5495b-1d93-472b-8a50-ce80a5f72ff6'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:19:47.008406 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-71c5495b-1d93-472b-8a50-ce80a5f72ff6 admin admin] The token cache is being invalidate because project b887285389c04d2cb53ccf0820374e54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81771) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:19:47.008703 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-71c5495b-1d93-472b-8a50-ce80a5f72ff6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:19:47.009412 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-71c5495b-1d93-472b-8a50-ce80a5f72ff6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:19:47.011184 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1091/2205] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:19:46 2022] DELETE /identity/v3/projects/b887285389c04d2cb53ccf0820374e54 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:19:47.017483 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c5ef51-ac65-4e25-9176-cb21f6d4710b None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:19:47.018199 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c5ef51-ac65-4e25-9176-cb21f6d4710b None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:19:47.018199 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c5ef51-ac65-4e25-9176-cb21f6d4710b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:19:47.033721 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:19:47.042039 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-a9c5ef51-ac65-4e25-9176-cb21f6d4710b None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:19:47.045277 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-a9c5ef51-ac65-4e25-9176-cb21f6d4710b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:19:47.133962 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1115/2206] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:19:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:19:47.136893 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9800d84-031c-42c3-89bf-6894ad5201d2 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:19:47.137354 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9800d84-031c-42c3-89bf-6894ad5201d2 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:19:47.137827 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9800d84-031c-42c3-89bf-6894ad5201d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:19:47.165370 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-d9800d84-031c-42c3-89bf-6894ad5201d2 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:19:47.166818 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-d9800d84-031c-42c3-89bf-6894ad5201d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:19:47.181067 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1092/2207] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:19:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:19:47.185072 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c5ef51-ac65-4e25-9176-cb21f6d4710b None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:19:47.204598 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c5ef51-ac65-4e25-9176-cb21f6d4710b None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:19:47.213324 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a9d57d-1a92-490f-9e99-ed2eff648098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:19:47.214599 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a9d57d-1a92-490f-9e99-ed2eff648098 admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:19:47.215132 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a9d57d-1a92-490f-9e99-ed2eff648098 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:19:47.215667 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a9d57d-1a92-490f-9e99-ed2eff648098 admin admin] PATH_INFO: `/v3/domains` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:19:47.216898 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a9d57d-1a92-490f-9e99-ed2eff648098 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:19:47.217420 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a9d57d-1a92-490f-9e99-ed2eff648098 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:19:47.221493 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a9d57d-1a92-490f-9e99-ed2eff648098 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:19:47.245574 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1116/2208] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 6 20:19:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:19:47.251579 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9800d84-031c-42c3-89bf-6894ad5201d2 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:19:47.270010 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9800d84-031c-42c3-89bf-6894ad5201d2 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:19:47.276798 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79851093-46e9-449b-93b4-8cc9feff10a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:19:47.277644 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79851093-46e9-449b-93b4-8cc9feff10a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:19:47.278171 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79851093-46e9-449b-93b4-8cc9feff10a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:19:47.278912 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79851093-46e9-449b-93b4-8cc9feff10a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:19:47.279766 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79851093-46e9-449b-93b4-8cc9feff10a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:19:47.283048 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79851093-46e9-449b-93b4-8cc9feff10a0 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:19:47.319036 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1093/2209] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:19:47 2022] POST /identity/v3/projects => generated 391 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:19:47.324020 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a9d57d-1a92-490f-9e99-ed2eff648098 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:19:47.335430 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a9d57d-1a92-490f-9e99-ed2eff648098 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:19:47.407329 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db31f75-8f60-4da4-a797-151e160287e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:19:47.408124 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db31f75-8f60-4da4-a797-151e160287e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:19:47.408477 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db31f75-8f60-4da4-a797-151e160287e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:19:47.408960 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db31f75-8f60-4da4-a797-151e160287e0 admin admin] PATH_INFO: `/v3/users` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:19:47.409761 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db31f75-8f60-4da4-a797-151e160287e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:19:47.411927 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db31f75-8f60-4da4-a797-151e160287e0 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:19:47.444540 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1117/2210] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:19:47 2022] POST /identity/v3/users => generated 296 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:19:47.451376 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79851093-46e9-449b-93b4-8cc9feff10a0 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:19:47.471185 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79851093-46e9-449b-93b4-8cc9feff10a0 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:19:47.479803 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376832f8-581f-422e-82ec-1ebd89e8db87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:19:47.481853 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376832f8-581f-422e-82ec-1ebd89e8db87 admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:19:47.481853 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376832f8-581f-422e-82ec-1ebd89e8db87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:19:47.482345 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376832f8-581f-422e-82ec-1ebd89e8db87 admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:19:47.483127 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376832f8-581f-422e-82ec-1ebd89e8db87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:19:47.486983 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376832f8-581f-422e-82ec-1ebd89e8db87 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:19:47.508471 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1094/2211] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:19:47 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:19:47.512666 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db31f75-8f60-4da4-a797-151e160287e0 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:19:47.525570 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db31f75-8f60-4da4-a797-151e160287e0 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:19:47.532161 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de1cda0-0b48-4453-8fcb-d469d2f6f240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:19:47.533087 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de1cda0-0b48-4453-8fcb-d469d2f6f240 admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:19:47.533468 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de1cda0-0b48-4453-8fcb-d469d2f6f240 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:19:47.533928 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de1cda0-0b48-4453-8fcb-d469d2f6f240 admin admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:19:47.534902 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de1cda0-0b48-4453-8fcb-d469d2f6f240 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:19:47.537316 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de1cda0-0b48-4453-8fcb-d469d2f6f240 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:19:47.552902 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1118/2212] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:19:47 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 20:19:47.557347 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376832f8-581f-422e-82ec-1ebd89e8db87 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:19:47.569779 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376832f8-581f-422e-82ec-1ebd89e8db87 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:19:47.575520 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7020b3e-ea4e-451e-8bb4-13e2b3143cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:19:47.576371 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7020b3e-ea4e-451e-8bb4-13e2b3143cfc admin admin] REQUEST_METHOD: `PUT` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:19:47.576697 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7020b3e-ea4e-451e-8bb4-13e2b3143cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:19:47.576981 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7020b3e-ea4e-451e-8bb4-13e2b3143cfc admin admin] PATH_INFO: `/v3/projects/5406554b5c4b4b629133ba4d32d18e29/users/7ef33ae1f4154964a3a9fccea1b29e15/roles/4eea39e24b06431aa357053fe7907a3d` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:19:47.598668 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7020b3e-ea4e-451e-8bb4-13e2b3143cfc admin admin] RBAC: Authorizing `identity:create_grant(project_id=5406554b5c4b4b629133ba4d32d18e29, user_id=7ef33ae1f4154964a3a9fccea1b29e15, role_id=4eea39e24b06431aa357053fe7907a3d)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:19:47.601805 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7020b3e-ea4e-451e-8bb4-13e2b3143cfc admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:19:47.605708 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:19:47.626189 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1095/2213] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:19:47 2022] PUT /identity/v3/projects/5406554b5c4b4b629133ba4d32d18e29/users/7ef33ae1f4154964a3a9fccea1b29e15/roles/4eea39e24b06431aa357053fe7907a3d => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 20:19:47.637927 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f882d63-f18c-480c-aa4d-9aecb4afd359 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:19:47.638369 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f882d63-f18c-480c-aa4d-9aecb4afd359 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:19:47.638813 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f882d63-f18c-480c-aa4d-9aecb4afd359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:19:47.679173 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-3f882d63-f18c-480c-aa4d-9aecb4afd359 None None] MFA Rules not processed for user `7ef33ae1f4154964a3a9fccea1b29e15`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:19:47.680622 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-3f882d63-f18c-480c-aa4d-9aecb4afd359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:19:47.764631 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1119/2214] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:19:47 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:19:47.777024 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7020b3e-ea4e-451e-8bb4-13e2b3143cfc admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:19:47.779454 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-b7020b3e-ea4e-451e-8bb4-13e2b3143cfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:19:47.873488 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7020b3e-ea4e-451e-8bb4-13e2b3143cfc admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:19:47.879274 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f71873-bdc6-426b-abb0-877e6cc3d0eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df1bca993bc142908cc7cf85eabd2b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:19:47.880370 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f71873-bdc6-426b-abb0-877e6cc3d0eb service cinder] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:19:47.881032 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f71873-bdc6-426b-abb0-877e6cc3d0eb service cinder] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:19:47.881568 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f71873-bdc6-426b-abb0-877e6cc3d0eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:19:47.897214 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f71873-bdc6-426b-abb0-877e6cc3d0eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:19:47.899410 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f71873-bdc6-426b-abb0-877e6cc3d0eb service cinder] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:19:47.919133 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1096/2215] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:19:47 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:19:57.761837 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=96,keystone:INSERT=3 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:19:57.908292 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=28 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:20:06.065731 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f882d63-f18c-480c-aa4d-9aecb4afd359 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:20:06.068332 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-3f882d63-f18c-480c-aa4d-9aecb4afd359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:20:06.190518 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f882d63-f18c-480c-aa4d-9aecb4afd359 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:20:06.197202 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff99cb8-240f-4dec-9adc-b34e9b19cbc5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5776cbf4f6714b2083c3ebdd6d98cc87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:20:06.197986 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff99cb8-240f-4dec-9adc-b34e9b19cbc5 service nova] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:20:06.198469 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff99cb8-240f-4dec-9adc-b34e9b19cbc5 service nova] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:20:06.198678 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff99cb8-240f-4dec-9adc-b34e9b19cbc5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:20:06.200594 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-6ff99cb8-240f-4dec-9adc-b34e9b19cbc5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:20:06.288802 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff99cb8-240f-4dec-9adc-b34e9b19cbc5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:20:06.291925 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff99cb8-240f-4dec-9adc-b34e9b19cbc5 service nova] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:20:06.305325 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1120/2216] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:20:06 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:20:09.152998 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f71873-bdc6-426b-abb0-877e6cc3d0eb service cinder] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:20:09.157289 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-f7f71873-bdc6-426b-abb0-877e6cc3d0eb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:20:09.358803 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f71873-bdc6-426b-abb0-877e6cc3d0eb service cinder] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:20:09.384689 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aaa408-cb58-42f3-ae03-1d05608d84c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:20:09.385894 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aaa408-cb58-42f3-ae03-1d05608d84c8 service neutron] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:20:09.386979 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aaa408-cb58-42f3-ae03-1d05608d84c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:20:09.387309 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aaa408-cb58-42f3-ae03-1d05608d84c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:20:09.390317 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-93aaa408-cb58-42f3-ae03-1d05608d84c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:20:09.408672 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93aaa408-cb58-42f3-ae03-1d05608d84c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:20:09.411452 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93aaa408-cb58-42f3-ae03-1d05608d84c8 service neutron] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:20:09.432963 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1097/2217] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:20:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:20:10.158971 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff99cb8-240f-4dec-9adc-b34e9b19cbc5 service nova] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:20:10.191335 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff99cb8-240f-4dec-9adc-b34e9b19cbc5 service nova] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:20:10.217050 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6403db3e-64a0-4d7c-8886-73fc14f5e936 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:20:10.223341 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6403db3e-64a0-4d7c-8886-73fc14f5e936 service neutron] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:20:10.224050 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6403db3e-64a0-4d7c-8886-73fc14f5e936 service neutron] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:20:10.224600 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6403db3e-64a0-4d7c-8886-73fc14f5e936 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:20:10.235634 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-6403db3e-64a0-4d7c-8886-73fc14f5e936 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:20:10.446997 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6403db3e-64a0-4d7c-8886-73fc14f5e936 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:20:10.452105 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6403db3e-64a0-4d7c-8886-73fc14f5e936 service neutron] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:20:10.500956 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1121/2218] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:20:10 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:20:12.041091 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aaa408-cb58-42f3-ae03-1d05608d84c8 service neutron] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:20:12.232818 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aaa408-cb58-42f3-ae03-1d05608d84c8 service neutron] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:20:12.243768 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c57d22-cc9d-4577-8a66-53d7a33a052e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:20:12.245199 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c57d22-cc9d-4577-8a66-53d7a33a052e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:20:12.245899 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c57d22-cc9d-4577-8a66-53d7a33a052e admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:20:12.246466 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c57d22-cc9d-4577-8a66-53d7a33a052e admin admin] PATH_INFO: `/v3/users/7ef33ae1f4154964a3a9fccea1b29e15` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:20:12.252338 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c57d22-cc9d-4577-8a66-53d7a33a052e admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ef33ae1f4154964a3a9fccea1b29e15)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:20:12.256863 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c57d22-cc9d-4577-8a66-53d7a33a052e admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:20:12.370032 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-b2c57d22-cc9d-4577-8a66-53d7a33a052e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ef33ae1f4154964a3a9fccea1b29e15'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:20:12.386181 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-b2c57d22-cc9d-4577-8a66-53d7a33a052e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ef33ae1f4154964a3a9fccea1b29e15'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:20:12.387106 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-b2c57d22-cc9d-4577-8a66-53d7a33a052e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ef33ae1f4154964a3a9fccea1b29e15'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:20:12.387933 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-b2c57d22-cc9d-4577-8a66-53d7a33a052e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ef33ae1f4154964a3a9fccea1b29e15'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:20:12.400444 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-b2c57d22-cc9d-4577-8a66-53d7a33a052e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ef33ae1f4154964a3a9fccea1b29e15'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:20:12.433922 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1098/2219] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:20:12 2022] DELETE /identity/v3/users/7ef33ae1f4154964a3a9fccea1b29e15 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:20:12.449698 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e813357-715c-49f5-8db0-fe884bc22ef6 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:20:12.457112 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e813357-715c-49f5-8db0-fe884bc22ef6 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:20:12.463454 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e813357-715c-49f5-8db0-fe884bc22ef6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:20:12.557813 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-1e813357-715c-49f5-8db0-fe884bc22ef6 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:20:12.560340 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-1e813357-715c-49f5-8db0-fe884bc22ef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:20:12.680589 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:20:12.776940 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1122/2220] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:20:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:20:12.798763 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c57d22-cc9d-4577-8a66-53d7a33a052e admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:20:12.803719 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-b2c57d22-cc9d-4577-8a66-53d7a33a052e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:20:12.888981 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:20:12.962739 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c57d22-cc9d-4577-8a66-53d7a33a052e admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:20:12.978087 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed7e60a-857b-46e6-90bc-28a4be94fb96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:20:12.978087 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed7e60a-857b-46e6-90bc-28a4be94fb96 service neutron] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:20:12.978087 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed7e60a-857b-46e6-90bc-28a4be94fb96 service neutron] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:20:12.978087 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed7e60a-857b-46e6-90bc-28a4be94fb96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:20:12.987909 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed7e60a-857b-46e6-90bc-28a4be94fb96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:20:12.990617 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed7e60a-857b-46e6-90bc-28a4be94fb96 service neutron] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:20:13.011411 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1099/2221] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:20:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 20:20:13.906677 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e813357-715c-49f5-8db0-fe884bc22ef6 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:20:13.909003 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-1e813357-715c-49f5-8db0-fe884bc22ef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:20:13.927918 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e813357-715c-49f5-8db0-fe884bc22ef6 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:20:13.937819 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79d594f-2a5c-4fba-a25b-6aff0c168360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:20:13.938591 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79d594f-2a5c-4fba-a25b-6aff0c168360 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:20:13.938958 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79d594f-2a5c-4fba-a25b-6aff0c168360 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:20:13.939451 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79d594f-2a5c-4fba-a25b-6aff0c168360 admin admin] PATH_INFO: `/v3/projects/5406554b5c4b4b629133ba4d32d18e29` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:20:13.940961 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79d594f-2a5c-4fba-a25b-6aff0c168360 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5406554b5c4b4b629133ba4d32d18e29)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:20:13.943807 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79d594f-2a5c-4fba-a25b-6aff0c168360 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:20:14.019123 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-b79d594f-2a5c-4fba-a25b-6aff0c168360 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5406554b5c4b4b629133ba4d32d18e29', 'request_id': 'req-b79d594f-2a5c-4fba-a25b-6aff0c168360'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:20:14.032281 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-b79d594f-2a5c-4fba-a25b-6aff0c168360 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5406554b5c4b4b629133ba4d32d18e29', 'request_id': 'req-b79d594f-2a5c-4fba-a25b-6aff0c168360'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:20:14.043222 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-b79d594f-2a5c-4fba-a25b-6aff0c168360 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5406554b5c4b4b629133ba4d32d18e29', 'request_id': 'req-b79d594f-2a5c-4fba-a25b-6aff0c168360'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:20:14.060616 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-b79d594f-2a5c-4fba-a25b-6aff0c168360 admin admin] The token cache is being invalidate because project 5406554b5c4b4b629133ba4d32d18e29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81772) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:20:14.060910 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-b79d594f-2a5c-4fba-a25b-6aff0c168360 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:20:14.061974 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-b79d594f-2a5c-4fba-a25b-6aff0c168360 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:20:14.064130 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1123/2222] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:20:13 2022] DELETE /identity/v3/projects/5406554b5c4b4b629133ba4d32d18e29 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:20:14.068952 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774f9730-7372-4367-94aa-3c0abd340ea2 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:20:14.070449 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774f9730-7372-4367-94aa-3c0abd340ea2 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:20:14.071026 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774f9730-7372-4367-94aa-3c0abd340ea2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:20:14.113532 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-774f9730-7372-4367-94aa-3c0abd340ea2 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:20:14.115083 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-774f9730-7372-4367-94aa-3c0abd340ea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:20:14.205122 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1100/2223] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:20:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:20:14.214220 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683fedcd-f52b-448a-906a-46028728cb67 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:20:14.214691 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683fedcd-f52b-448a-906a-46028728cb67 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:20:14.215072 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683fedcd-f52b-448a-906a-46028728cb67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:20:14.239460 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-683fedcd-f52b-448a-906a-46028728cb67 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:20:14.240798 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-683fedcd-f52b-448a-906a-46028728cb67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:20:14.249686 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1124/2224] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:20:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:20:14.255599 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774f9730-7372-4367-94aa-3c0abd340ea2 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:20:14.267620 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774f9730-7372-4367-94aa-3c0abd340ea2 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:20:14.273507 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3319a218-cc7e-4270-8c69-291f288fa91b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:20:14.274278 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3319a218-cc7e-4270-8c69-291f288fa91b admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:20:14.274633 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3319a218-cc7e-4270-8c69-291f288fa91b admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:20:14.275053 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3319a218-cc7e-4270-8c69-291f288fa91b admin admin] PATH_INFO: `/v3/domains` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:20:14.275768 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3319a218-cc7e-4270-8c69-291f288fa91b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:20:14.276148 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3319a218-cc7e-4270-8c69-291f288fa91b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:20:14.278042 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3319a218-cc7e-4270-8c69-291f288fa91b admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:20:14.297918 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1101/2225] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 6 20:20:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:20:14.299903 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683fedcd-f52b-448a-906a-46028728cb67 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:20:14.312256 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683fedcd-f52b-448a-906a-46028728cb67 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:20:14.317976 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583ed094-e3e5-4883-8abb-c3f55d9bf9e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:20:14.318815 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583ed094-e3e5-4883-8abb-c3f55d9bf9e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:20:14.319099 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583ed094-e3e5-4883-8abb-c3f55d9bf9e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:20:14.319416 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583ed094-e3e5-4883-8abb-c3f55d9bf9e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:20:14.320269 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583ed094-e3e5-4883-8abb-c3f55d9bf9e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:20:14.322303 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583ed094-e3e5-4883-8abb-c3f55d9bf9e1 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:20:14.360301 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1125/2226] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:20:14 2022] POST /identity/v3/projects => generated 383 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:20:14.365050 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3319a218-cc7e-4270-8c69-291f288fa91b admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:20:14.378101 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3319a218-cc7e-4270-8c69-291f288fa91b admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:20:14.488150 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa967da7-45c9-4b23-9bbd-18a1de27ee76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:20:14.489026 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa967da7-45c9-4b23-9bbd-18a1de27ee76 admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:20:14.490048 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa967da7-45c9-4b23-9bbd-18a1de27ee76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:20:14.490048 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa967da7-45c9-4b23-9bbd-18a1de27ee76 admin admin] PATH_INFO: `/v3/users` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:20:14.491059 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa967da7-45c9-4b23-9bbd-18a1de27ee76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:20:14.492663 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa967da7-45c9-4b23-9bbd-18a1de27ee76 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:20:14.517422 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583ed094-e3e5-4883-8abb-c3f55d9bf9e1 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:20:14.523248 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-583ed094-e3e5-4883-8abb-c3f55d9bf9e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:20:14.549057 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1102/2227] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:20:14 2022] POST /identity/v3/users => generated 292 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:20:14.550870 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa967da7-45c9-4b23-9bbd-18a1de27ee76 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:20:14.564894 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa967da7-45c9-4b23-9bbd-18a1de27ee76 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:20:14.571302 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcffa90-8292-4f8d-8e1d-610442889932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:20:14.572121 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcffa90-8292-4f8d-8e1d-610442889932 admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:20:14.572554 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcffa90-8292-4f8d-8e1d-610442889932 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:20:14.572913 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcffa90-8292-4f8d-8e1d-610442889932 admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:20:14.573742 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcffa90-8292-4f8d-8e1d-610442889932 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:20:14.575973 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcffa90-8292-4f8d-8e1d-610442889932 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:20:14.595861 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1103/2228] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:20:14 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:20:14.597709 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcffa90-8292-4f8d-8e1d-610442889932 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:20:14.619510 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcffa90-8292-4f8d-8e1d-610442889932 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:20:14.628037 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ae7070-e05f-4d34-94e7-db0485cee6b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:20:14.628696 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ae7070-e05f-4d34-94e7-db0485cee6b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:20:14.629005 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ae7070-e05f-4d34-94e7-db0485cee6b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:20:14.629282 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ae7070-e05f-4d34-94e7-db0485cee6b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:20:14.629993 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ae7070-e05f-4d34-94e7-db0485cee6b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:20:14.631989 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ae7070-e05f-4d34-94e7-db0485cee6b5 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:20:14.651193 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1104/2229] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:20:14 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:20:14.652773 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ae7070-e05f-4d34-94e7-db0485cee6b5 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:20:14.676634 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ae7070-e05f-4d34-94e7-db0485cee6b5 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:20:14.688583 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f80b8ba-720a-4e33-bf6f-88e042b6ab39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:20:14.690848 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f80b8ba-720a-4e33-bf6f-88e042b6ab39 admin admin] REQUEST_METHOD: `PUT` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:20:14.691538 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f80b8ba-720a-4e33-bf6f-88e042b6ab39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:20:14.692584 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f80b8ba-720a-4e33-bf6f-88e042b6ab39 admin admin] PATH_INFO: `/v3/projects/48d7280f7ee04d389193da7bf05eaf1a/users/a385e6b1f23d449f8738a6f87720184d/roles/4eea39e24b06431aa357053fe7907a3d` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:20:14.727043 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f80b8ba-720a-4e33-bf6f-88e042b6ab39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48d7280f7ee04d389193da7bf05eaf1a, user_id=a385e6b1f23d449f8738a6f87720184d, role_id=4eea39e24b06431aa357053fe7907a3d)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:20:14.730414 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f80b8ba-720a-4e33-bf6f-88e042b6ab39 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:20:14.733495 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583ed094-e3e5-4883-8abb-c3f55d9bf9e1 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:20:14.735515 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:20:14.741535 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc78cefb-224e-4fda-a8f8-bbda096f47f1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5776cbf4f6714b2083c3ebdd6d98cc87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:20:14.742283 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc78cefb-224e-4fda-a8f8-bbda096f47f1 service nova] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:20:14.742627 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc78cefb-224e-4fda-a8f8-bbda096f47f1 service nova] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:20:14.742838 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc78cefb-224e-4fda-a8f8-bbda096f47f1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:20:14.745373 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-bc78cefb-224e-4fda-a8f8-bbda096f47f1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:20:14.752505 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1105/2230] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:20:14 2022] PUT /identity/v3/projects/48d7280f7ee04d389193da7bf05eaf1a/users/a385e6b1f23d449f8738a6f87720184d/roles/4eea39e24b06431aa357053fe7907a3d => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:20:14.758090 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c3a6f2-4419-4076-8910-83578520d048 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:20:14.758581 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c3a6f2-4419-4076-8910-83578520d048 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:20:14.758938 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c3a6f2-4419-4076-8910-83578520d048 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:20:14.777032 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:20:14.785152 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-f3c3a6f2-4419-4076-8910-83578520d048 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:20:14.788208 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-f3c3a6f2-4419-4076-8910-83578520d048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:20:14.843841 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc78cefb-224e-4fda-a8f8-bbda096f47f1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:20:14.849300 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc78cefb-224e-4fda-a8f8-bbda096f47f1 service nova] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:20:14.875763 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1126/2231] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:20:14 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:20:14.925042 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1106/2232] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:20:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:20:14.936950 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc78cefb-224e-4fda-a8f8-bbda096f47f1 service nova] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:20:14.947608 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-bc78cefb-224e-4fda-a8f8-bbda096f47f1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:20:15.041393 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc78cefb-224e-4fda-a8f8-bbda096f47f1 service nova] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:20:15.134737 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a626aef3-9e1a-4c08-92bd-8113cc125317 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:20:15.136309 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a626aef3-9e1a-4c08-92bd-8113cc125317 service neutron] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:20:15.136309 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a626aef3-9e1a-4c08-92bd-8113cc125317 service neutron] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:20:15.136309 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a626aef3-9e1a-4c08-92bd-8113cc125317 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:20:15.156181 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a626aef3-9e1a-4c08-92bd-8113cc125317 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:20:15.160073 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a626aef3-9e1a-4c08-92bd-8113cc125317 service neutron] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:20:15.203663 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1127/2233] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:20:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:20:16.724287 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c3a6f2-4419-4076-8910-83578520d048 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:20:16.726915 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-f3c3a6f2-4419-4076-8910-83578520d048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:20:16.778909 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c3a6f2-4419-4076-8910-83578520d048 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:20:16.795924 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60df686a-5f43-48c8-8e9c-4f6da1f2f60a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:20:16.798529 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60df686a-5f43-48c8-8e9c-4f6da1f2f60a service neutron] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:20:16.800206 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60df686a-5f43-48c8-8e9c-4f6da1f2f60a service neutron] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:20:16.802714 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60df686a-5f43-48c8-8e9c-4f6da1f2f60a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:20:16.810744 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-60df686a-5f43-48c8-8e9c-4f6da1f2f60a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:20:17.025867 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60df686a-5f43-48c8-8e9c-4f6da1f2f60a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:20:17.029572 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60df686a-5f43-48c8-8e9c-4f6da1f2f60a service neutron] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:20:17.082487 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1107/2234] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:20:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:20:18.878070 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5543a3ce-36bd-4c3e-9bd5-aa9de11f1eb8 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:20:18.879084 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5543a3ce-36bd-4c3e-9bd5-aa9de11f1eb8 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:20:18.879084 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5543a3ce-36bd-4c3e-9bd5-aa9de11f1eb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:20:18.914993 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-5543a3ce-36bd-4c3e-9bd5-aa9de11f1eb8 None None] MFA Rules not processed for user `ecde3e4bfd8b4f3490edd3832cea9e48`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:20:18.916732 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-5543a3ce-36bd-4c3e-9bd5-aa9de11f1eb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:20:19.053852 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1128/2235] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:20:18 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:20:24.185053 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60df686a-5f43-48c8-8e9c-4f6da1f2f60a service neutron] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:20:24.239782 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60df686a-5f43-48c8-8e9c-4f6da1f2f60a service neutron] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:20:24.264690 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a015d2a-a221-4feb-9524-9d57663220fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:20:24.270087 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a015d2a-a221-4feb-9524-9d57663220fa service neutron] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:20:24.271416 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a015d2a-a221-4feb-9524-9d57663220fa service neutron] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:20:24.272209 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a015d2a-a221-4feb-9524-9d57663220fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:20:24.278428 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-8a015d2a-a221-4feb-9524-9d57663220fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:20:24.534416 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a015d2a-a221-4feb-9524-9d57663220fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:20:24.536174 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a015d2a-a221-4feb-9524-9d57663220fa service neutron] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:20:24.550301 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1108/2236] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:20:24 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:20:26.989828 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a667682-b093-44da-8f53-081624b92fe8 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:20:26.990658 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a667682-b093-44da-8f53-081624b92fe8 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:20:26.990658 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a667682-b093-44da-8f53-081624b92fe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:20:27.085172 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-0a667682-b093-44da-8f53-081624b92fe8 None None] MFA Rules not processed for user `a385e6b1f23d449f8738a6f87720184d`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:20:27.088268 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-0a667682-b093-44da-8f53-081624b92fe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:20:27.139637 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:20:27.302458 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1129/2237] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:20:26 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:20:27.310469 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a015d2a-a221-4feb-9524-9d57663220fa service neutron] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:20:27.335467 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a015d2a-a221-4feb-9524-9d57663220fa service neutron] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:20:27.345152 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba5a4c8-20fb-41e4-a691-ea4d59a74300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:20:27.346816 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba5a4c8-20fb-41e4-a691-ea4d59a74300 admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:20:27.346998 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba5a4c8-20fb-41e4-a691-ea4d59a74300 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:20:27.347472 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba5a4c8-20fb-41e4-a691-ea4d59a74300 admin admin] PATH_INFO: `/v3/projects` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:20:27.348689 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba5a4c8-20fb-41e4-a691-ea4d59a74300 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:20:27.352843 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba5a4c8-20fb-41e4-a691-ea4d59a74300 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:20:27.397260 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:20:27.452940 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1109/2238] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:20:27 2022] POST /identity/v3/projects => generated 383 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:20:27.460327 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a667682-b093-44da-8f53-081624b92fe8 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:20:27.489923 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a667682-b093-44da-8f53-081624b92fe8 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:20:27.623293 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add5a91a-fb31-452e-834a-a1b964175ddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:20:27.625123 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add5a91a-fb31-452e-834a-a1b964175ddb admin admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:20:27.626176 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add5a91a-fb31-452e-834a-a1b964175ddb admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:20:27.626730 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add5a91a-fb31-452e-834a-a1b964175ddb admin admin] PATH_INFO: `/v3/users` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:20:27.627999 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add5a91a-fb31-452e-834a-a1b964175ddb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:20:27.631088 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add5a91a-fb31-452e-834a-a1b964175ddb admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:20:27.709514 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba5a4c8-20fb-41e4-a691-ea4d59a74300 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:20:27.712146 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1130/2239] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:20:27 2022] POST /identity/v3/users => generated 298 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:20:27.737155 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba5a4c8-20fb-41e4-a691-ea4d59a74300 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:20:27.748001 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f26a8b-a95b-4205-a03d-7825521228de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:20:27.748801 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f26a8b-a95b-4205-a03d-7825521228de admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:20:27.749155 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f26a8b-a95b-4205-a03d-7825521228de admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:20:27.749517 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f26a8b-a95b-4205-a03d-7825521228de admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:20:27.751401 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f26a8b-a95b-4205-a03d-7825521228de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:20:27.754208 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f26a8b-a95b-4205-a03d-7825521228de admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:20:27.816909 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1110/2240] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:20:27 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:20:27.823319 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add5a91a-fb31-452e-834a-a1b964175ddb admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:20:27.848988 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add5a91a-fb31-452e-834a-a1b964175ddb admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:20:27.868033 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed424bf-b4de-4df1-9b3a-062a5ca561d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:20:27.868793 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed424bf-b4de-4df1-9b3a-062a5ca561d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:20:27.869121 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed424bf-b4de-4df1-9b3a-062a5ca561d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:20:27.869638 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed424bf-b4de-4df1-9b3a-062a5ca561d6 admin admin] PATH_INFO: `/v3/projects/94f1527ff6a24463b13a1936a5ac57cd/users/6ed33b2c77ed40659aad1827865939c5/roles/e0ab41d8ae22474688b5361bcc75e0f0` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:20:27.922716 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed424bf-b4de-4df1-9b3a-062a5ca561d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94f1527ff6a24463b13a1936a5ac57cd, user_id=6ed33b2c77ed40659aad1827865939c5, role_id=e0ab41d8ae22474688b5361bcc75e0f0)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:20:27.928284 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed424bf-b4de-4df1-9b3a-062a5ca561d6 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:20:27.970951 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1131/2241] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:20:27 2022] PUT /identity/v3/projects/94f1527ff6a24463b13a1936a5ac57cd/users/6ed33b2c77ed40659aad1827865939c5/roles/e0ab41d8ae22474688b5361bcc75e0f0 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:20:27.972765 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f26a8b-a95b-4205-a03d-7825521228de admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:20:27.993402 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f26a8b-a95b-4205-a03d-7825521228de admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:20:28.166974 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bb3f07-22de-4c56-96f7-6cbbb341cadd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:20:28.168514 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bb3f07-22de-4c56-96f7-6cbbb341cadd admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:20:28.169176 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bb3f07-22de-4c56-96f7-6cbbb341cadd admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:20:28.169872 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bb3f07-22de-4c56-96f7-6cbbb341cadd admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:20:28.171507 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bb3f07-22de-4c56-96f7-6cbbb341cadd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:20:28.175766 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bb3f07-22de-4c56-96f7-6cbbb341cadd admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:20:28.202572 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1111/2242] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:20:27 2022] GET /identity/v3/roles => generated 1420 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:20:28.210872 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed424bf-b4de-4df1-9b3a-062a5ca561d6 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:20:28.230899 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed424bf-b4de-4df1-9b3a-062a5ca561d6 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:20:28.248024 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0b25cc-e996-4ddb-8dfc-6da16f63914f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:20:28.249794 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0b25cc-e996-4ddb-8dfc-6da16f63914f admin admin] REQUEST_METHOD: `PUT` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:20:28.250704 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0b25cc-e996-4ddb-8dfc-6da16f63914f admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:20:28.251417 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0b25cc-e996-4ddb-8dfc-6da16f63914f admin admin] PATH_INFO: `/v3/projects/94f1527ff6a24463b13a1936a5ac57cd/users/6ed33b2c77ed40659aad1827865939c5/roles/e0ab41d8ae22474688b5361bcc75e0f0` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:20:28.254837 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0b25cc-e996-4ddb-8dfc-6da16f63914f admin admin] RBAC: Authorizing `identity:create_grant(project_id=94f1527ff6a24463b13a1936a5ac57cd, user_id=6ed33b2c77ed40659aad1827865939c5, role_id=e0ab41d8ae22474688b5361bcc75e0f0)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:20:28.261346 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0b25cc-e996-4ddb-8dfc-6da16f63914f admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:20:28.289545 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1132/2243] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:20:28 2022] PUT /identity/v3/projects/94f1527ff6a24463b13a1936a5ac57cd/users/6ed33b2c77ed40659aad1827865939c5/roles/e0ab41d8ae22474688b5361bcc75e0f0 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:20:35.529652 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bb3f07-22de-4c56-96f7-6cbbb341cadd admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:20:35.805133 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bb3f07-22de-4c56-96f7-6cbbb341cadd admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:20:36.015592 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22ca2bf-327d-46c2-bb55-812886c4b196 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:20:36.016777 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22ca2bf-327d-46c2-bb55-812886c4b196 service neutron] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:20:36.017143 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22ca2bf-327d-46c2-bb55-812886c4b196 service neutron] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:20:36.021399 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22ca2bf-327d-46c2-bb55-812886c4b196 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:20:36.033794 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-c22ca2bf-327d-46c2-bb55-812886c4b196 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:20:36.143266 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22ca2bf-327d-46c2-bb55-812886c4b196 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:20:36.158752 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22ca2bf-327d-46c2-bb55-812886c4b196 service neutron] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:20:36.187180 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1112/2244] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:20:35 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 658 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:20:37.212689 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf116cc-eea1-4c30-bbfb-43325568e772 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:20:37.214127 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf116cc-eea1-4c30-bbfb-43325568e772 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:20:37.217179 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf116cc-eea1-4c30-bbfb-43325568e772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:20:37.348862 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-5cf116cc-eea1-4c30-bbfb-43325568e772 None None] MFA Rules not processed for user `6ed33b2c77ed40659aad1827865939c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:20:37.352771 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-5cf116cc-eea1-4c30-bbfb-43325568e772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:20:37.601675 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:20:37.676882 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1133/2245] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:20:37 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 473 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:20:37.701432 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22ca2bf-327d-46c2-bb55-812886c4b196 service neutron] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:20:37.703553 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-c22ca2bf-327d-46c2-bb55-812886c4b196 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:20:37.866327 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:20:37.893990 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22ca2bf-327d-46c2-bb55-812886c4b196 service neutron] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:20:37.901648 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e2c477-f24d-4be1-b050-21e77685f965 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07af18f35ffc42a295278d5349e06381', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:20:37.903089 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e2c477-f24d-4be1-b050-21e77685f965 service glance] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:20:37.905252 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e2c477-f24d-4be1-b050-21e77685f965 service glance] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:20:37.906146 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e2c477-f24d-4be1-b050-21e77685f965 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:20:38.080211 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf116cc-eea1-4c30-bbfb-43325568e772 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:20:38.082370 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-5cf116cc-eea1-4c30-bbfb-43325568e772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:20:38.107410 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e2c477-f24d-4be1-b050-21e77685f965 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:20:38.110488 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e2c477-f24d-4be1-b050-21e77685f965 service glance] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:20:38.146396 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1113/2246] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:20:37 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:20:38.334491 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf116cc-eea1-4c30-bbfb-43325568e772 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:20:38.359619 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56a446d-dcf0-4585-a96c-cdea094ecc92 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5776cbf4f6714b2083c3ebdd6d98cc87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:20:38.360691 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56a446d-dcf0-4585-a96c-cdea094ecc92 service nova] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:20:38.361240 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56a446d-dcf0-4585-a96c-cdea094ecc92 service nova] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:20:38.361810 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56a446d-dcf0-4585-a96c-cdea094ecc92 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:20:38.364574 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-d56a446d-dcf0-4585-a96c-cdea094ecc92 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:20:38.475768 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ec2b0c-88aa-4bf1-940a-776c98f0b4df None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:20:38.476230 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ec2b0c-88aa-4bf1-940a-776c98f0b4df None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:20:38.476708 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ec2b0c-88aa-4bf1-940a-776c98f0b4df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:20:38.495495 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56a446d-dcf0-4585-a96c-cdea094ecc92 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:20:38.497407 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56a446d-dcf0-4585-a96c-cdea094ecc92 service nova] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:20:38.512705 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-29ec2b0c-88aa-4bf1-940a-776c98f0b4df None None] MFA Rules not processed for user `1a52117df70447dba9bbee35d083e44f`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:20:38.515092 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-29ec2b0c-88aa-4bf1-940a-776c98f0b4df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:20:38.525544 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1134/2247] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:20:38 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:20:38.653477 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1114/2248] 10.222.0.20 () {58 vars in 1009 bytes} [Tue Sep 6 20:20:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:20:38.680519 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56a446d-dcf0-4585-a96c-cdea094ecc92 service nova] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:20:38.682916 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-d56a446d-dcf0-4585-a96c-cdea094ecc92 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:20:38.821103 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56a446d-dcf0-4585-a96c-cdea094ecc92 service nova] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:20:38.853354 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1a6812-1f09-4ccc-956e-dec44456667d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b72516236da47e292d78de201b0552b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:20:38.854066 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1a6812-1f09-4ccc-956e-dec44456667d service swift] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:20:38.854673 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1a6812-1f09-4ccc-956e-dec44456667d service swift] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:20:38.855003 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1a6812-1f09-4ccc-956e-dec44456667d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:20:38.871803 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1a6812-1f09-4ccc-956e-dec44456667d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:20:38.876413 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1a6812-1f09-4ccc-956e-dec44456667d service swift] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:20:38.895983 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1135/2249] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Sep 6 20:20:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 20:20:40.038284 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d560ed-d2c8-40c9-9b14-66f9e4cbb3ae None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:20:40.039124 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d560ed-d2c8-40c9-9b14-66f9e4cbb3ae None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:20:40.039124 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d560ed-d2c8-40c9-9b14-66f9e4cbb3ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:20:40.092387 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-54d560ed-d2c8-40c9-9b14-66f9e4cbb3ae None None] MFA Rules not processed for user `ad400ce73b644938aef19e2da731c4dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:20:40.095356 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-54d560ed-d2c8-40c9-9b14-66f9e4cbb3ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:20:40.305101 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1115/2250] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:20:40 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:20:40.894370 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1a6812-1f09-4ccc-956e-dec44456667d service swift] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:20:40.921428 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1a6812-1f09-4ccc-956e-dec44456667d service swift] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:20:40.928522 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b3c1ea-f1cf-4d87-8815-b171ddbfac26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:20:40.930416 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b3c1ea-f1cf-4d87-8815-b171ddbfac26 service neutron] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:20:40.931117 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b3c1ea-f1cf-4d87-8815-b171ddbfac26 service neutron] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:20:40.931763 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b3c1ea-f1cf-4d87-8815-b171ddbfac26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:20:40.935078 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-96b3c1ea-f1cf-4d87-8815-b171ddbfac26 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:20:40.948106 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b3c1ea-f1cf-4d87-8815-b171ddbfac26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:20:40.950062 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b3c1ea-f1cf-4d87-8815-b171ddbfac26 service neutron] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:20:40.967428 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1136/2251] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:20:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:20:41.732182 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d560ed-d2c8-40c9-9b14-66f9e4cbb3ae None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:20:41.736732 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-54d560ed-d2c8-40c9-9b14-66f9e4cbb3ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:20:41.968459 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:20:42.001818 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d560ed-d2c8-40c9-9b14-66f9e4cbb3ae None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:20:42.008968 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b0df34-6afb-4c59-8f19-2feec13871f2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f57872de7bb74ea48d30e93e51830aba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:20:42.009803 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b0df34-6afb-4c59-8f19-2feec13871f2 service placement] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:20:42.010090 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b0df34-6afb-4c59-8f19-2feec13871f2 service placement] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:20:42.010392 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b0df34-6afb-4c59-8f19-2feec13871f2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:20:42.013866 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-79b0df34-6afb-4c59-8f19-2feec13871f2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:20:42.062539 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b0df34-6afb-4c59-8f19-2feec13871f2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:20:42.069593 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b0df34-6afb-4c59-8f19-2feec13871f2 service placement] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:20:42.103061 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1116/2252] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:20:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:20:48.109479 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b3c1ea-f1cf-4d87-8815-b171ddbfac26 service neutron] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:20:48.194917 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b3c1ea-f1cf-4d87-8815-b171ddbfac26 service neutron] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:20:48.216768 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126ff34d-56b4-4833-8201-e9eaabe92bf3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5776cbf4f6714b2083c3ebdd6d98cc87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:20:48.217986 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126ff34d-56b4-4833-8201-e9eaabe92bf3 service nova] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:20:48.218925 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126ff34d-56b4-4833-8201-e9eaabe92bf3 service nova] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:20:48.222164 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126ff34d-56b4-4833-8201-e9eaabe92bf3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:20:48.228957 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-126ff34d-56b4-4833-8201-e9eaabe92bf3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:20:48.267035 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:20:48.282415 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126ff34d-56b4-4833-8201-e9eaabe92bf3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:20:48.288880 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126ff34d-56b4-4833-8201-e9eaabe92bf3 service nova] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:20:48.313174 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1137/2253] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:20:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:20:52.089344 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=11 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:20:55.077095 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801ddbf0-d63e-4903-b9e6-2690f81adf82 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:20:55.078284 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801ddbf0-d63e-4903-b9e6-2690f81adf82 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:20:55.078762 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801ddbf0-d63e-4903-b9e6-2690f81adf82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:20:55.113845 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-801ddbf0-d63e-4903-b9e6-2690f81adf82 None None] MFA Rules not processed for user `a385e6b1f23d449f8738a6f87720184d`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:20:55.115503 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-801ddbf0-d63e-4903-b9e6-2690f81adf82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:20:55.124281 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1117/2254] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:20:55 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:20:55.528841 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81526bf1-068d-46ca-9b3a-f27f8ad5b32d None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:20:55.528841 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81526bf1-068d-46ca-9b3a-f27f8ad5b32d None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:20:55.529405 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81526bf1-068d-46ca-9b3a-f27f8ad5b32d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:20:55.529951 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801ddbf0-d63e-4903-b9e6-2690f81adf82 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:20:55.533094 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-801ddbf0-d63e-4903-b9e6-2690f81adf82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:20:55.588327 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-81526bf1-068d-46ca-9b3a-f27f8ad5b32d None None] MFA Rules not processed for user `7be957f2c15c4f6f949b3f284a30624d`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:20:55.590620 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-81526bf1-068d-46ca-9b3a-f27f8ad5b32d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:20:55.609116 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1138/2255] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:20:55 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:20:55.686511 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801ddbf0-d63e-4903-b9e6-2690f81adf82 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:20:55.698211 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd2f1e3-f6d7-40e1-a2c3-e28daa35e2ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:20:55.699511 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd2f1e3-f6d7-40e1-a2c3-e28daa35e2ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:20:55.700222 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd2f1e3-f6d7-40e1-a2c3-e28daa35e2ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:20:55.700913 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd2f1e3-f6d7-40e1-a2c3-e28daa35e2ac admin admin] PATH_INFO: `/v3/users/87f8cc79586f4cb79c962ede8342f580` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:20:55.703210 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd2f1e3-f6d7-40e1-a2c3-e28daa35e2ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=87f8cc79586f4cb79c962ede8342f580)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:20:55.707808 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd2f1e3-f6d7-40e1-a2c3-e28daa35e2ac admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:20:55.798810 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81526bf1-068d-46ca-9b3a-f27f8ad5b32d None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:20:55.801755 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-cfd2f1e3-f6d7-40e1-a2c3-e28daa35e2ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87f8cc79586f4cb79c962ede8342f580'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:20:55.817562 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-cfd2f1e3-f6d7-40e1-a2c3-e28daa35e2ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87f8cc79586f4cb79c962ede8342f580'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:20:55.818318 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-cfd2f1e3-f6d7-40e1-a2c3-e28daa35e2ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87f8cc79586f4cb79c962ede8342f580'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:20:55.819095 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-cfd2f1e3-f6d7-40e1-a2c3-e28daa35e2ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87f8cc79586f4cb79c962ede8342f580'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:20:55.819473 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81526bf1-068d-46ca-9b3a-f27f8ad5b32d None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:20:55.828069 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077ea1fd-7811-4ddc-9f96-3dbfe2250061 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:20:55.830324 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077ea1fd-7811-4ddc-9f96-3dbfe2250061 service neutron] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:20:55.831052 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077ea1fd-7811-4ddc-9f96-3dbfe2250061 service neutron] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:20:55.831703 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077ea1fd-7811-4ddc-9f96-3dbfe2250061 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:20:55.832690 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-cfd2f1e3-f6d7-40e1-a2c3-e28daa35e2ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87f8cc79586f4cb79c962ede8342f580'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:20:55.835554 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-077ea1fd-7811-4ddc-9f96-3dbfe2250061 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:20:55.855233 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077ea1fd-7811-4ddc-9f96-3dbfe2250061 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:20:55.858964 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077ea1fd-7811-4ddc-9f96-3dbfe2250061 service neutron] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:20:55.870609 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1118/2256] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:20:55 2022] DELETE /identity/v3/users/87f8cc79586f4cb79c962ede8342f580 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 20:20:55.878752 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1139/2257] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:20:55 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 20:20:56.650033 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd2f1e3-f6d7-40e1-a2c3-e28daa35e2ac admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:20:56.653555 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-cfd2f1e3-f6d7-40e1-a2c3-e28daa35e2ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:20:56.866574 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd2f1e3-f6d7-40e1-a2c3-e28daa35e2ac admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:20:56.879230 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6bd26c-e46e-4d59-8dd8-220c3c2da504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:20:56.880080 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6bd26c-e46e-4d59-8dd8-220c3c2da504 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:20:56.880372 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6bd26c-e46e-4d59-8dd8-220c3c2da504 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:20:56.880654 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6bd26c-e46e-4d59-8dd8-220c3c2da504 admin admin] PATH_INFO: `/v3/projects/b5de971d8037434fbaf3c5aa93729028` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:20:56.882358 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6bd26c-e46e-4d59-8dd8-220c3c2da504 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5de971d8037434fbaf3c5aa93729028)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:20:56.884585 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6bd26c-e46e-4d59-8dd8-220c3c2da504 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:20:56.932056 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:20:57.056868 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-df6bd26c-e46e-4d59-8dd8-220c3c2da504 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5de971d8037434fbaf3c5aa93729028', 'request_id': 'req-df6bd26c-e46e-4d59-8dd8-220c3c2da504'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:20:57.081098 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-df6bd26c-e46e-4d59-8dd8-220c3c2da504 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5de971d8037434fbaf3c5aa93729028', 'request_id': 'req-df6bd26c-e46e-4d59-8dd8-220c3c2da504'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:20:57.098741 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-df6bd26c-e46e-4d59-8dd8-220c3c2da504 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5de971d8037434fbaf3c5aa93729028', 'request_id': 'req-df6bd26c-e46e-4d59-8dd8-220c3c2da504'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:20:57.123228 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-df6bd26c-e46e-4d59-8dd8-220c3c2da504 admin admin] The token cache is being invalidate because project b5de971d8037434fbaf3c5aa93729028 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81771) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:20:57.123755 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-df6bd26c-e46e-4d59-8dd8-220c3c2da504 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:20:57.124987 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-df6bd26c-e46e-4d59-8dd8-220c3c2da504 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:20:57.128625 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1119/2258] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:20:56 2022] DELETE /identity/v3/projects/b5de971d8037434fbaf3c5aa93729028 => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:20:57.480479 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077ea1fd-7811-4ddc-9f96-3dbfe2250061 service neutron] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:20:57.482557 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-077ea1fd-7811-4ddc-9f96-3dbfe2250061 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:20:57.523556 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077ea1fd-7811-4ddc-9f96-3dbfe2250061 service neutron] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:20:57.646798 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882ba7ec-608c-442d-9eb4-19cd07dfcc2c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5776cbf4f6714b2083c3ebdd6d98cc87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:20:57.648063 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882ba7ec-608c-442d-9eb4-19cd07dfcc2c service nova] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:20:57.648700 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882ba7ec-608c-442d-9eb4-19cd07dfcc2c service nova] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:20:57.649123 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882ba7ec-608c-442d-9eb4-19cd07dfcc2c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:20:57.652574 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-882ba7ec-608c-442d-9eb4-19cd07dfcc2c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:20:57.676749 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882ba7ec-608c-442d-9eb4-19cd07dfcc2c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:20:57.682168 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882ba7ec-608c-442d-9eb4-19cd07dfcc2c service nova] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:20:57.708955 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1140/2259] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:20:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:21:02.601077 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f43eadf-c746-41e2-b120-cc5dbd59dcae None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:21:02.601947 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f43eadf-c746-41e2-b120-cc5dbd59dcae None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:21:02.602409 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f43eadf-c746-41e2-b120-cc5dbd59dcae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:21:02.678977 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-4f43eadf-c746-41e2-b120-cc5dbd59dcae None None] MFA Rules not processed for user `1a52117df70447dba9bbee35d083e44f`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:21:02.680391 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-4f43eadf-c746-41e2-b120-cc5dbd59dcae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:21:02.898557 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1120/2260] 10.222.0.20 () {58 vars in 1009 bytes} [Tue Sep 6 20:21:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:21:02.917937 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882ba7ec-608c-442d-9eb4-19cd07dfcc2c service nova] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:21:02.921584 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-882ba7ec-608c-442d-9eb4-19cd07dfcc2c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:21:02.922449 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f43eadf-c746-41e2-b120-cc5dbd59dcae None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:21:02.924532 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-4f43eadf-c746-41e2-b120-cc5dbd59dcae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:21:03.072095 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:21:03.100121 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f43eadf-c746-41e2-b120-cc5dbd59dcae None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:21:03.113288 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5534f668-89c0-4823-a8a2-8d523e04cfdc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b72516236da47e292d78de201b0552b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:21:03.113965 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5534f668-89c0-4823-a8a2-8d523e04cfdc service swift] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:21:03.114300 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5534f668-89c0-4823-a8a2-8d523e04cfdc service swift] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:21:03.114587 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5534f668-89c0-4823-a8a2-8d523e04cfdc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:21:03.128997 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5534f668-89c0-4823-a8a2-8d523e04cfdc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:21:03.131083 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5534f668-89c0-4823-a8a2-8d523e04cfdc service swift] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:21:03.159743 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1121/2261] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Sep 6 20:21:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 06 20:21:03.172793 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882ba7ec-608c-442d-9eb4-19cd07dfcc2c service nova] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:21:03.186711 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4571fcc9-6318-4eaf-a306-5f667d509dad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5776cbf4f6714b2083c3ebdd6d98cc87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:21:03.189158 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4571fcc9-6318-4eaf-a306-5f667d509dad service nova] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:21:03.189759 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4571fcc9-6318-4eaf-a306-5f667d509dad service nova] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:21:03.190349 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4571fcc9-6318-4eaf-a306-5f667d509dad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:21:03.193918 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-4571fcc9-6318-4eaf-a306-5f667d509dad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:21:03.213275 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4571fcc9-6318-4eaf-a306-5f667d509dad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:21:03.216288 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4571fcc9-6318-4eaf-a306-5f667d509dad service nova] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:21:03.243512 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1141/2262] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:21:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:21:13.144933 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=6 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:21:13.226669 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=74 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:21:29.720983 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cc9380-6f36-44fc-9f9d-f5052ffcff36 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:21:29.721682 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cc9380-6f36-44fc-9f9d-f5052ffcff36 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:21:29.721996 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cc9380-6f36-44fc-9f9d-f5052ffcff36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:21:29.746537 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-58cc9380-6f36-44fc-9f9d-f5052ffcff36 None None] MFA Rules not processed for user `f49ef54a38614cb880577626e65b0e32`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:21:29.748493 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-58cc9380-6f36-44fc-9f9d-f5052ffcff36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:21:29.866373 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1122/2263] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:21:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:21:39.846018 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=25 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:21:40.196402 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4571fcc9-6318-4eaf-a306-5f667d509dad service nova] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:21:40.198599 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-4571fcc9-6318-4eaf-a306-5f667d509dad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:21:40.322394 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4571fcc9-6318-4eaf-a306-5f667d509dad service nova] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:21:40.334912 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989a089b-5a12-4c5f-a66b-c18fa28a1ee7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:21:40.334912 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989a089b-5a12-4c5f-a66b-c18fa28a1ee7 service neutron] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:21:40.334912 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989a089b-5a12-4c5f-a66b-c18fa28a1ee7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:21:40.334912 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989a089b-5a12-4c5f-a66b-c18fa28a1ee7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:21:40.335561 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-989a089b-5a12-4c5f-a66b-c18fa28a1ee7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:21:40.438384 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989a089b-5a12-4c5f-a66b-c18fa28a1ee7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:21:40.441094 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989a089b-5a12-4c5f-a66b-c18fa28a1ee7 service neutron] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:21:40.457239 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1142/2264] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:21:40 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 20:21:42.599176 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cc9380-6f36-44fc-9f9d-f5052ffcff36 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:21:42.611228 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cc9380-6f36-44fc-9f9d-f5052ffcff36 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:21:42.615737 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d186cd8-e55c-486b-a811-6b9cbeaea188 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:21:42.616461 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d186cd8-e55c-486b-a811-6b9cbeaea188 service neutron] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:21:42.616752 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d186cd8-e55c-486b-a811-6b9cbeaea188 service neutron] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:21:42.617031 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d186cd8-e55c-486b-a811-6b9cbeaea188 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:21:42.619670 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-3d186cd8-e55c-486b-a811-6b9cbeaea188 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:21:42.774695 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d186cd8-e55c-486b-a811-6b9cbeaea188 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:21:42.782201 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d186cd8-e55c-486b-a811-6b9cbeaea188 service neutron] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:21:42.811686 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1123/2265] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:21:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:21:50.161133 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989a089b-5a12-4c5f-a66b-c18fa28a1ee7 service neutron] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:21:50.162731 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-989a089b-5a12-4c5f-a66b-c18fa28a1ee7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:21:50.193166 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989a089b-5a12-4c5f-a66b-c18fa28a1ee7 service neutron] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:21:50.198246 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef8f8c7-6093-44f0-b307-1665b5bd4a5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:21:50.198899 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef8f8c7-6093-44f0-b307-1665b5bd4a5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:21:50.199206 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef8f8c7-6093-44f0-b307-1665b5bd4a5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:21:50.200312 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef8f8c7-6093-44f0-b307-1665b5bd4a5e admin admin] PATH_INFO: `/v3/users/ecde3e4bfd8b4f3490edd3832cea9e48` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:21:50.201937 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef8f8c7-6093-44f0-b307-1665b5bd4a5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ecde3e4bfd8b4f3490edd3832cea9e48)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:21:50.203937 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef8f8c7-6093-44f0-b307-1665b5bd4a5e admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:21:50.297078 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-8ef8f8c7-6093-44f0-b307-1665b5bd4a5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ecde3e4bfd8b4f3490edd3832cea9e48'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:21:50.298084 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-8ef8f8c7-6093-44f0-b307-1665b5bd4a5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ecde3e4bfd8b4f3490edd3832cea9e48'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:21:50.325851 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-8ef8f8c7-6093-44f0-b307-1665b5bd4a5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ecde3e4bfd8b4f3490edd3832cea9e48'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:21:50.326440 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-8ef8f8c7-6093-44f0-b307-1665b5bd4a5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ecde3e4bfd8b4f3490edd3832cea9e48'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:21:50.342125 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-8ef8f8c7-6093-44f0-b307-1665b5bd4a5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ecde3e4bfd8b4f3490edd3832cea9e48'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:21:50.376372 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1143/2266] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:21:50 2022] DELETE /identity/v3/users/ecde3e4bfd8b4f3490edd3832cea9e48 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:21:50.701764 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d186cd8-e55c-486b-a811-6b9cbeaea188 service neutron] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:21:50.731781 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d186cd8-e55c-486b-a811-6b9cbeaea188 service neutron] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:21:50.868059 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ab2b1d-4186-4bd5-858e-b84b4690828d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:21:50.869183 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ab2b1d-4186-4bd5-858e-b84b4690828d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:21:50.869951 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ab2b1d-4186-4bd5-858e-b84b4690828d admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:21:50.870465 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ab2b1d-4186-4bd5-858e-b84b4690828d admin admin] PATH_INFO: `/v3/projects/b005ff023a9f46fda3ceef605996c385` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:21:50.873267 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ab2b1d-4186-4bd5-858e-b84b4690828d admin admin] RBAC: Authorizing `identity:delete_project(project_id=b005ff023a9f46fda3ceef605996c385)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:21:50.877373 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ab2b1d-4186-4bd5-858e-b84b4690828d admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:21:50.977996 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-15ab2b1d-4186-4bd5-858e-b84b4690828d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b005ff023a9f46fda3ceef605996c385', 'request_id': 'req-15ab2b1d-4186-4bd5-858e-b84b4690828d'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:21:50.992454 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-15ab2b1d-4186-4bd5-858e-b84b4690828d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b005ff023a9f46fda3ceef605996c385', 'request_id': 'req-15ab2b1d-4186-4bd5-858e-b84b4690828d'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:21:51.008659 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-15ab2b1d-4186-4bd5-858e-b84b4690828d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b005ff023a9f46fda3ceef605996c385', 'request_id': 'req-15ab2b1d-4186-4bd5-858e-b84b4690828d'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:21:51.018345 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:21:51.035433 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-15ab2b1d-4186-4bd5-858e-b84b4690828d admin admin] The token cache is being invalidate because project b005ff023a9f46fda3ceef605996c385 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81771) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:21:51.035953 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-15ab2b1d-4186-4bd5-858e-b84b4690828d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:21:51.037429 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-15ab2b1d-4186-4bd5-858e-b84b4690828d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:21:51.043286 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1124/2267] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:21:50 2022] DELETE /identity/v3/projects/b005ff023a9f46fda3ceef605996c385 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 20:21:51.049551 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747e9cd2-ee86-4043-9f09-98da6201a929 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:21:51.050041 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747e9cd2-ee86-4043-9f09-98da6201a929 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:21:51.050481 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747e9cd2-ee86-4043-9f09-98da6201a929 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:21:51.059737 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:21:51.171596 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-747e9cd2-ee86-4043-9f09-98da6201a929 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:21:51.173965 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-747e9cd2-ee86-4043-9f09-98da6201a929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:21:51.295462 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1144/2268] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:21:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:21:51.303710 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8397461-5bf4-4d82-95cb-12a443b0f2e4 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:21:51.304082 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8397461-5bf4-4d82-95cb-12a443b0f2e4 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:21:51.304409 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8397461-5bf4-4d82-95cb-12a443b0f2e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:21:51.341189 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-a8397461-5bf4-4d82-95cb-12a443b0f2e4 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:21:51.343596 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-a8397461-5bf4-4d82-95cb-12a443b0f2e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:21:51.360149 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1125/2269] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:21:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:21:51.368543 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747e9cd2-ee86-4043-9f09-98da6201a929 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:21:51.386393 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747e9cd2-ee86-4043-9f09-98da6201a929 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:21:51.397296 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c566ab32-337d-439f-8b0a-4b0c832447b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:21:51.398390 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c566ab32-337d-439f-8b0a-4b0c832447b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:21:51.399157 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c566ab32-337d-439f-8b0a-4b0c832447b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:21:51.399684 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c566ab32-337d-439f-8b0a-4b0c832447b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:21:51.401098 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c566ab32-337d-439f-8b0a-4b0c832447b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:21:51.401663 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c566ab32-337d-439f-8b0a-4b0c832447b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:21:51.403975 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c566ab32-337d-439f-8b0a-4b0c832447b5 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:21:51.420372 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1145/2270] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 6 20:21:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:21:51.426391 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8397461-5bf4-4d82-95cb-12a443b0f2e4 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:21:51.444700 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8397461-5bf4-4d82-95cb-12a443b0f2e4 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:21:51.454357 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3295cabd-c948-4ce3-b8b4-c7b569220a8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:21:51.456090 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3295cabd-c948-4ce3-b8b4-c7b569220a8e admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:21:51.456587 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3295cabd-c948-4ce3-b8b4-c7b569220a8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:21:51.457065 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3295cabd-c948-4ce3-b8b4-c7b569220a8e admin admin] PATH_INFO: `/v3/projects` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:21:51.458359 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3295cabd-c948-4ce3-b8b4-c7b569220a8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:21:51.462304 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3295cabd-c948-4ce3-b8b4-c7b569220a8e admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:21:51.530422 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1126/2271] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:21:51 2022] POST /identity/v3/projects => generated 365 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:21:51.531997 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c566ab32-337d-439f-8b0a-4b0c832447b5 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:21:51.544654 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c566ab32-337d-439f-8b0a-4b0c832447b5 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:21:51.630564 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd23033-6cb9-4a13-b517-3009abf7cfde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:21:51.631189 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd23033-6cb9-4a13-b517-3009abf7cfde admin admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:21:51.631471 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd23033-6cb9-4a13-b517-3009abf7cfde admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:21:51.631839 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd23033-6cb9-4a13-b517-3009abf7cfde admin admin] PATH_INFO: `/v3/users` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:21:51.632485 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd23033-6cb9-4a13-b517-3009abf7cfde admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:21:51.634505 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd23033-6cb9-4a13-b517-3009abf7cfde admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:21:51.668633 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1146/2272] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:21:51 2022] POST /identity/v3/users => generated 283 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:21:51.670750 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3295cabd-c948-4ce3-b8b4-c7b569220a8e admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:21:51.687942 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3295cabd-c948-4ce3-b8b4-c7b569220a8e admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:21:51.694620 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69cd1e3-4f7a-45fc-bc89-242bf628a870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:21:51.695312 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69cd1e3-4f7a-45fc-bc89-242bf628a870 admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:21:51.695706 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69cd1e3-4f7a-45fc-bc89-242bf628a870 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:21:51.696122 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69cd1e3-4f7a-45fc-bc89-242bf628a870 admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:21:51.697085 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69cd1e3-4f7a-45fc-bc89-242bf628a870 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:21:51.699591 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69cd1e3-4f7a-45fc-bc89-242bf628a870 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:21:51.718514 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1127/2273] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:21:51 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:21:51.723453 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd23033-6cb9-4a13-b517-3009abf7cfde admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:21:51.739349 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd23033-6cb9-4a13-b517-3009abf7cfde admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:21:51.746365 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ac8d03-c317-4bee-be79-4accaae9ad7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:21:51.747660 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ac8d03-c317-4bee-be79-4accaae9ad7d admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:21:51.748117 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ac8d03-c317-4bee-be79-4accaae9ad7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:21:51.748564 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ac8d03-c317-4bee-be79-4accaae9ad7d admin admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:21:51.749368 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ac8d03-c317-4bee-be79-4accaae9ad7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:21:51.751757 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ac8d03-c317-4bee-be79-4accaae9ad7d admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:21:51.768086 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1147/2274] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:21:51 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 20:21:51.772849 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69cd1e3-4f7a-45fc-bc89-242bf628a870 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:21:51.786765 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69cd1e3-4f7a-45fc-bc89-242bf628a870 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:21:51.793541 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb92c660-46ff-4c9d-abe8-97f71b205fbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:21:51.794586 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb92c660-46ff-4c9d-abe8-97f71b205fbb admin admin] REQUEST_METHOD: `PUT` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:21:51.795008 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb92c660-46ff-4c9d-abe8-97f71b205fbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:21:51.795434 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb92c660-46ff-4c9d-abe8-97f71b205fbb admin admin] PATH_INFO: `/v3/projects/6e4a163c51f54694a6dfeeb4d77d4a25/users/325d321df79f4b53962002a6895a1505/roles/4eea39e24b06431aa357053fe7907a3d` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:21:51.825727 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb92c660-46ff-4c9d-abe8-97f71b205fbb admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e4a163c51f54694a6dfeeb4d77d4a25, user_id=325d321df79f4b53962002a6895a1505, role_id=4eea39e24b06431aa357053fe7907a3d)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:21:51.831043 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb92c660-46ff-4c9d-abe8-97f71b205fbb admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:21:51.860404 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1128/2275] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:21:51 2022] PUT /identity/v3/projects/6e4a163c51f54694a6dfeeb4d77d4a25/users/325d321df79f4b53962002a6895a1505/roles/4eea39e24b06431aa357053fe7907a3d => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 20:21:51.879641 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9559d647-9b48-465c-9ec7-abb936fca184 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:21:51.880135 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9559d647-9b48-465c-9ec7-abb936fca184 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:21:51.880629 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9559d647-9b48-465c-9ec7-abb936fca184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:21:51.922824 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-9559d647-9b48-465c-9ec7-abb936fca184 None None] MFA Rules not processed for user `325d321df79f4b53962002a6895a1505`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:21:51.925069 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-9559d647-9b48-465c-9ec7-abb936fca184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:21:51.964457 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:21:52.032558 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1148/2276] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:21:51 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:21:52.037192 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb92c660-46ff-4c9d-abe8-97f71b205fbb admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:21:52.155488 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb92c660-46ff-4c9d-abe8-97f71b205fbb admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:21:52.160205 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a89c61-3cf7-4778-99c5-a87aca3657f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:21:52.160957 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a89c61-3cf7-4778-99c5-a87aca3657f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:21:52.161151 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a89c61-3cf7-4778-99c5-a87aca3657f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:21:52.161462 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a89c61-3cf7-4778-99c5-a87aca3657f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:21:52.162383 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a89c61-3cf7-4778-99c5-a87aca3657f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:21:52.164428 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a89c61-3cf7-4778-99c5-a87aca3657f9 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:21:52.201162 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1129/2277] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:21:52 2022] POST /identity/v3/projects => generated 365 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 20:21:52.206105 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9559d647-9b48-465c-9ec7-abb936fca184 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:21:52.219294 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9559d647-9b48-465c-9ec7-abb936fca184 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:21:52.290439 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6307a0-12c2-4eda-89d2-975d48b427cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:21:52.291126 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6307a0-12c2-4eda-89d2-975d48b427cc admin admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:21:52.291528 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6307a0-12c2-4eda-89d2-975d48b427cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:21:52.291877 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6307a0-12c2-4eda-89d2-975d48b427cc admin admin] PATH_INFO: `/v3/users` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:21:52.292645 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6307a0-12c2-4eda-89d2-975d48b427cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:21:52.295618 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6307a0-12c2-4eda-89d2-975d48b427cc admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:21:52.327257 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1149/2278] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:21:52 2022] POST /identity/v3/users => generated 289 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:21:52.331961 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a89c61-3cf7-4778-99c5-a87aca3657f9 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:21:52.344060 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a89c61-3cf7-4778-99c5-a87aca3657f9 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:21:52.349881 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e917d1b0-5383-46c0-b555-b54a7be1fb2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:21:52.350654 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e917d1b0-5383-46c0-b555-b54a7be1fb2b admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:21:52.350978 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e917d1b0-5383-46c0-b555-b54a7be1fb2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:21:52.351425 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e917d1b0-5383-46c0-b555-b54a7be1fb2b admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:21:52.352277 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e917d1b0-5383-46c0-b555-b54a7be1fb2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:21:52.354128 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e917d1b0-5383-46c0-b555-b54a7be1fb2b admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:21:52.367290 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1130/2279] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:21:52 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 20:21:52.373106 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6307a0-12c2-4eda-89d2-975d48b427cc admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:21:52.384788 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6307a0-12c2-4eda-89d2-975d48b427cc admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:21:52.390247 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec26f3ae-2932-40b7-81bc-1920c9fc97ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:21:52.390985 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec26f3ae-2932-40b7-81bc-1920c9fc97ef admin admin] REQUEST_METHOD: `PUT` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:21:52.391309 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec26f3ae-2932-40b7-81bc-1920c9fc97ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:21:52.391622 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec26f3ae-2932-40b7-81bc-1920c9fc97ef admin admin] PATH_INFO: `/v3/projects/279d15d24d8f494a8a050f38d79c3349/users/45e22708b84e46d1bb90509bee489504/roles/e0ab41d8ae22474688b5361bcc75e0f0` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:21:52.413264 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec26f3ae-2932-40b7-81bc-1920c9fc97ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=279d15d24d8f494a8a050f38d79c3349, user_id=45e22708b84e46d1bb90509bee489504, role_id=e0ab41d8ae22474688b5361bcc75e0f0)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:21:52.416799 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec26f3ae-2932-40b7-81bc-1920c9fc97ef admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:21:52.438621 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1150/2280] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:21:52 2022] PUT /identity/v3/projects/279d15d24d8f494a8a050f38d79c3349/users/45e22708b84e46d1bb90509bee489504/roles/e0ab41d8ae22474688b5361bcc75e0f0 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 20:21:52.445359 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e917d1b0-5383-46c0-b555-b54a7be1fb2b admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:21:52.457570 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e917d1b0-5383-46c0-b555-b54a7be1fb2b admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:21:52.523790 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:21:52.534864 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf53ba61-15fb-4d7c-aa0e-9ee1efed40b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:21:52.535635 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf53ba61-15fb-4d7c-aa0e-9ee1efed40b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:21:52.535967 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf53ba61-15fb-4d7c-aa0e-9ee1efed40b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:21:52.536354 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf53ba61-15fb-4d7c-aa0e-9ee1efed40b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:21:52.537074 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf53ba61-15fb-4d7c-aa0e-9ee1efed40b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:21:52.538998 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf53ba61-15fb-4d7c-aa0e-9ee1efed40b8 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:21:52.551622 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1131/2281] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:21:52 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:21:52.556588 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec26f3ae-2932-40b7-81bc-1920c9fc97ef admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:21:52.568977 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec26f3ae-2932-40b7-81bc-1920c9fc97ef admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:21:52.574305 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616f095c-6f94-4f16-816f-496389fb5728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:21:52.575142 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616f095c-6f94-4f16-816f-496389fb5728 admin admin] REQUEST_METHOD: `PUT` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:21:52.575440 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616f095c-6f94-4f16-816f-496389fb5728 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:21:52.575821 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616f095c-6f94-4f16-816f-496389fb5728 admin admin] PATH_INFO: `/v3/projects/279d15d24d8f494a8a050f38d79c3349/users/45e22708b84e46d1bb90509bee489504/roles/e0ab41d8ae22474688b5361bcc75e0f0` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:21:52.578778 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616f095c-6f94-4f16-816f-496389fb5728 admin admin] RBAC: Authorizing `identity:create_grant(project_id=279d15d24d8f494a8a050f38d79c3349, user_id=45e22708b84e46d1bb90509bee489504, role_id=e0ab41d8ae22474688b5361bcc75e0f0)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:21:52.581575 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616f095c-6f94-4f16-816f-496389fb5728 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:21:52.594388 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1151/2282] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:21:52 2022] PUT /identity/v3/projects/279d15d24d8f494a8a050f38d79c3349/users/45e22708b84e46d1bb90509bee489504/roles/e0ab41d8ae22474688b5361bcc75e0f0 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:21:52.605052 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212c2a88-f5f9-4d41-b10e-4bc360867e98 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:21:52.605467 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212c2a88-f5f9-4d41-b10e-4bc360867e98 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:21:52.605737 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212c2a88-f5f9-4d41-b10e-4bc360867e98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:21:52.645743 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-212c2a88-f5f9-4d41-b10e-4bc360867e98 None None] MFA Rules not processed for user `45e22708b84e46d1bb90509bee489504`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:21:52.647678 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-212c2a88-f5f9-4d41-b10e-4bc360867e98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:21:52.790574 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1132/2283] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:21:52 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:21:52.804695 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616f095c-6f94-4f16-816f-496389fb5728 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:21:52.806888 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-616f095c-6f94-4f16-816f-496389fb5728 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:21:52.912030 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616f095c-6f94-4f16-816f-496389fb5728 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:21:52.920693 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cf1ba4-1940-4afa-a02b-14ba0d5a1f52 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df1bca993bc142908cc7cf85eabd2b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:21:52.920927 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cf1ba4-1940-4afa-a02b-14ba0d5a1f52 service cinder] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:21:52.921270 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cf1ba4-1940-4afa-a02b-14ba0d5a1f52 service cinder] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:21:52.921758 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cf1ba4-1940-4afa-a02b-14ba0d5a1f52 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:21:52.935613 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cf1ba4-1940-4afa-a02b-14ba0d5a1f52 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:21:52.937320 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cf1ba4-1940-4afa-a02b-14ba0d5a1f52 service cinder] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:21:52.952132 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1152/2284] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:21:52 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:21:53.517996 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212c2a88-f5f9-4d41-b10e-4bc360867e98 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:21:53.606919 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212c2a88-f5f9-4d41-b10e-4bc360867e98 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:21:53.612645 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7927bc6-ac4b-4f70-afa7-cb27bdf3faa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:21:53.613436 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7927bc6-ac4b-4f70-afa7-cb27bdf3faa3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:21:53.613863 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7927bc6-ac4b-4f70-afa7-cb27bdf3faa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:21:53.614246 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7927bc6-ac4b-4f70-afa7-cb27bdf3faa3 admin admin] PATH_INFO: `/v3/users/325d321df79f4b53962002a6895a1505` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:21:53.615861 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7927bc6-ac4b-4f70-afa7-cb27bdf3faa3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=325d321df79f4b53962002a6895a1505)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:21:53.618516 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7927bc6-ac4b-4f70-afa7-cb27bdf3faa3 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:21:53.691168 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-b7927bc6-ac4b-4f70-afa7-cb27bdf3faa3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '325d321df79f4b53962002a6895a1505'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:21:53.704147 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-b7927bc6-ac4b-4f70-afa7-cb27bdf3faa3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '325d321df79f4b53962002a6895a1505'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:21:53.705346 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-b7927bc6-ac4b-4f70-afa7-cb27bdf3faa3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '325d321df79f4b53962002a6895a1505'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:21:53.706431 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-b7927bc6-ac4b-4f70-afa7-cb27bdf3faa3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '325d321df79f4b53962002a6895a1505'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:21:53.717343 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-b7927bc6-ac4b-4f70-afa7-cb27bdf3faa3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '325d321df79f4b53962002a6895a1505'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:21:53.735970 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:21:53.744235 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1133/2285] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:21:53 2022] DELETE /identity/v3/users/325d321df79f4b53962002a6895a1505 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:21:53.750835 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcb070f-4043-41e9-b43b-3200e423b129 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:21:53.751420 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcb070f-4043-41e9-b43b-3200e423b129 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:21:53.751649 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcb070f-4043-41e9-b43b-3200e423b129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:21:53.761404 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:21:53.780821 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-1dcb070f-4043-41e9-b43b-3200e423b129 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:21:53.782454 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-1dcb070f-4043-41e9-b43b-3200e423b129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:21:53.860196 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1153/2286] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:21:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:21:53.871215 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7927bc6-ac4b-4f70-afa7-cb27bdf3faa3 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:21:53.873287 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-b7927bc6-ac4b-4f70-afa7-cb27bdf3faa3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:21:53.969125 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7927bc6-ac4b-4f70-afa7-cb27bdf3faa3 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:21:53.976103 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8460a09f-3d6c-4f9c-9e31-830f066f389f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:21:53.977189 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8460a09f-3d6c-4f9c-9e31-830f066f389f service neutron] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:21:53.977723 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8460a09f-3d6c-4f9c-9e31-830f066f389f service neutron] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:21:53.978208 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8460a09f-3d6c-4f9c-9e31-830f066f389f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:21:53.997767 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8460a09f-3d6c-4f9c-9e31-830f066f389f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:21:54.000458 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8460a09f-3d6c-4f9c-9e31-830f066f389f service neutron] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:21:54.014600 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1134/2287] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:21:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:21:54.420044 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcb070f-4043-41e9-b43b-3200e423b129 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:21:54.422639 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-1dcb070f-4043-41e9-b43b-3200e423b129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:21:54.443239 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcb070f-4043-41e9-b43b-3200e423b129 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:21:54.450300 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf3e25e-354f-4d1a-b875-2577451212f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:21:54.450985 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf3e25e-354f-4d1a-b875-2577451212f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:21:54.451266 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf3e25e-354f-4d1a-b875-2577451212f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:21:54.452297 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf3e25e-354f-4d1a-b875-2577451212f6 admin admin] PATH_INFO: `/v3/projects/6e4a163c51f54694a6dfeeb4d77d4a25` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:21:54.454797 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf3e25e-354f-4d1a-b875-2577451212f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e4a163c51f54694a6dfeeb4d77d4a25)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:21:54.458728 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf3e25e-354f-4d1a-b875-2577451212f6 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:21:54.531070 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-8cf3e25e-354f-4d1a-b875-2577451212f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e4a163c51f54694a6dfeeb4d77d4a25', 'request_id': 'req-8cf3e25e-354f-4d1a-b875-2577451212f6'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:21:54.541919 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-8cf3e25e-354f-4d1a-b875-2577451212f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e4a163c51f54694a6dfeeb4d77d4a25', 'request_id': 'req-8cf3e25e-354f-4d1a-b875-2577451212f6'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:21:54.550580 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-8cf3e25e-354f-4d1a-b875-2577451212f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e4a163c51f54694a6dfeeb4d77d4a25', 'request_id': 'req-8cf3e25e-354f-4d1a-b875-2577451212f6'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:21:54.568139 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-8cf3e25e-354f-4d1a-b875-2577451212f6 admin admin] The token cache is being invalidate because project 6e4a163c51f54694a6dfeeb4d77d4a25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81772) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:21:54.568605 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-8cf3e25e-354f-4d1a-b875-2577451212f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:21:54.569502 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-8cf3e25e-354f-4d1a-b875-2577451212f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:21:54.571293 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1154/2288] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:21:54 2022] DELETE /identity/v3/projects/6e4a163c51f54694a6dfeeb4d77d4a25 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:21:54.575831 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8460a09f-3d6c-4f9c-9e31-830f066f389f service neutron] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:21:54.587737 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8460a09f-3d6c-4f9c-9e31-830f066f389f service neutron] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:21:54.662589 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6db3f74-7f82-4bd3-9c43-aad9359e8301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:21:54.663138 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6db3f74-7f82-4bd3-9c43-aad9359e8301 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:21:54.663453 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6db3f74-7f82-4bd3-9c43-aad9359e8301 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:21:54.663909 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6db3f74-7f82-4bd3-9c43-aad9359e8301 admin admin] PATH_INFO: `/v3/users/45e22708b84e46d1bb90509bee489504` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:21:54.665147 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6db3f74-7f82-4bd3-9c43-aad9359e8301 admin admin] RBAC: Authorizing `identity:delete_user(user_id=45e22708b84e46d1bb90509bee489504)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:21:54.667165 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6db3f74-7f82-4bd3-9c43-aad9359e8301 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:21:54.726835 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-a6db3f74-7f82-4bd3-9c43-aad9359e8301 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45e22708b84e46d1bb90509bee489504'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:21:54.737318 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-a6db3f74-7f82-4bd3-9c43-aad9359e8301 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45e22708b84e46d1bb90509bee489504'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:21:54.738251 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-a6db3f74-7f82-4bd3-9c43-aad9359e8301 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45e22708b84e46d1bb90509bee489504'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:21:54.739047 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-a6db3f74-7f82-4bd3-9c43-aad9359e8301 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45e22708b84e46d1bb90509bee489504'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:21:54.748914 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-a6db3f74-7f82-4bd3-9c43-aad9359e8301 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45e22708b84e46d1bb90509bee489504'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:21:54.769973 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1135/2289] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:21:54 2022] DELETE /identity/v3/users/45e22708b84e46d1bb90509bee489504 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 20:21:55.180942 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf3e25e-354f-4d1a-b875-2577451212f6 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:21:55.186008 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-8cf3e25e-354f-4d1a-b875-2577451212f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:21:55.232265 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf3e25e-354f-4d1a-b875-2577451212f6 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:21:55.329534 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ffacff-2982-4245-88e2-b7eb29ba05fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:21:55.330341 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ffacff-2982-4245-88e2-b7eb29ba05fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:21:55.330714 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ffacff-2982-4245-88e2-b7eb29ba05fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:21:55.331096 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ffacff-2982-4245-88e2-b7eb29ba05fa admin admin] PATH_INFO: `/v3/projects/279d15d24d8f494a8a050f38d79c3349` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:21:55.332511 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ffacff-2982-4245-88e2-b7eb29ba05fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=279d15d24d8f494a8a050f38d79c3349)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:21:55.334709 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ffacff-2982-4245-88e2-b7eb29ba05fa admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:21:55.379497 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:21:55.401448 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-69ffacff-2982-4245-88e2-b7eb29ba05fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '279d15d24d8f494a8a050f38d79c3349', 'request_id': 'req-69ffacff-2982-4245-88e2-b7eb29ba05fa'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:21:55.412625 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-69ffacff-2982-4245-88e2-b7eb29ba05fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '279d15d24d8f494a8a050f38d79c3349', 'request_id': 'req-69ffacff-2982-4245-88e2-b7eb29ba05fa'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:21:55.425303 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-69ffacff-2982-4245-88e2-b7eb29ba05fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '279d15d24d8f494a8a050f38d79c3349', 'request_id': 'req-69ffacff-2982-4245-88e2-b7eb29ba05fa'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:21:55.445814 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-69ffacff-2982-4245-88e2-b7eb29ba05fa admin admin] The token cache is being invalidate because project 279d15d24d8f494a8a050f38d79c3349 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81772) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:21:55.446236 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-69ffacff-2982-4245-88e2-b7eb29ba05fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:21:55.447344 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-69ffacff-2982-4245-88e2-b7eb29ba05fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:21:55.450099 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1155/2290] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:21:55 2022] DELETE /identity/v3/projects/279d15d24d8f494a8a050f38d79c3349 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:21:55.456143 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e637713e-865e-4560-b7c6-5496cab476ce None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:21:55.456713 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e637713e-865e-4560-b7c6-5496cab476ce None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:21:55.456835 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e637713e-865e-4560-b7c6-5496cab476ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:21:55.482044 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-e637713e-865e-4560-b7c6-5496cab476ce None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:21:55.483406 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-e637713e-865e-4560-b7c6-5496cab476ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:21:55.503365 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:21:55.574956 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1136/2291] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:21:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:21:55.588920 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0734195-aab4-4930-8c46-c0576a327eeb None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:21:55.589437 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0734195-aab4-4930-8c46-c0576a327eeb None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:21:55.590696 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0734195-aab4-4930-8c46-c0576a327eeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:21:55.625987 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-d0734195-aab4-4930-8c46-c0576a327eeb None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:21:55.627701 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-d0734195-aab4-4930-8c46-c0576a327eeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:21:55.639286 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1156/2292] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:21:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:21:55.645411 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e637713e-865e-4560-b7c6-5496cab476ce None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:21:55.660568 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e637713e-865e-4560-b7c6-5496cab476ce None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:21:55.667949 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e630ed4-8930-4b44-a0ea-2224fe481098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:21:55.669094 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e630ed4-8930-4b44-a0ea-2224fe481098 admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:21:55.669748 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e630ed4-8930-4b44-a0ea-2224fe481098 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:21:55.670119 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e630ed4-8930-4b44-a0ea-2224fe481098 admin admin] PATH_INFO: `/v3/domains` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:21:55.671333 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e630ed4-8930-4b44-a0ea-2224fe481098 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:21:55.671888 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e630ed4-8930-4b44-a0ea-2224fe481098 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:21:55.674661 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e630ed4-8930-4b44-a0ea-2224fe481098 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:21:55.696126 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1137/2293] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 6 20:21:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 20:21:55.702448 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0734195-aab4-4930-8c46-c0576a327eeb None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:21:55.716698 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0734195-aab4-4930-8c46-c0576a327eeb None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:21:55.724605 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2636bc3-ef92-44c7-9e9b-9d3b2938ac15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:21:55.726239 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2636bc3-ef92-44c7-9e9b-9d3b2938ac15 admin admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:21:55.726239 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2636bc3-ef92-44c7-9e9b-9d3b2938ac15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:21:55.727159 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2636bc3-ef92-44c7-9e9b-9d3b2938ac15 admin admin] PATH_INFO: `/v3/projects` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:21:55.727877 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2636bc3-ef92-44c7-9e9b-9d3b2938ac15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:21:55.730009 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2636bc3-ef92-44c7-9e9b-9d3b2938ac15 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:21:55.771766 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1157/2294] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:21:55 2022] POST /identity/v3/projects => generated 387 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 20:21:55.776427 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e630ed4-8930-4b44-a0ea-2224fe481098 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:21:55.790600 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e630ed4-8930-4b44-a0ea-2224fe481098 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:21:55.867456 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fad9e42-c6a0-40bf-a41f-689c6bd1d266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:21:55.868162 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fad9e42-c6a0-40bf-a41f-689c6bd1d266 admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:21:55.868442 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fad9e42-c6a0-40bf-a41f-689c6bd1d266 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:21:55.868714 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fad9e42-c6a0-40bf-a41f-689c6bd1d266 admin admin] PATH_INFO: `/v3/users` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:21:55.869367 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fad9e42-c6a0-40bf-a41f-689c6bd1d266 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:21:55.871625 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fad9e42-c6a0-40bf-a41f-689c6bd1d266 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:21:55.903676 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1138/2295] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:21:55 2022] POST /identity/v3/users => generated 294 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:21:55.909379 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2636bc3-ef92-44c7-9e9b-9d3b2938ac15 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:21:55.923084 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2636bc3-ef92-44c7-9e9b-9d3b2938ac15 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:21:55.928370 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdde7075-6cd4-4876-8f76-e0f312b9df3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:21:55.929034 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdde7075-6cd4-4876-8f76-e0f312b9df3f admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:21:55.929314 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdde7075-6cd4-4876-8f76-e0f312b9df3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:21:55.929590 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdde7075-6cd4-4876-8f76-e0f312b9df3f admin admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:21:55.930311 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdde7075-6cd4-4876-8f76-e0f312b9df3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:21:55.932005 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdde7075-6cd4-4876-8f76-e0f312b9df3f admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:21:55.948421 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1158/2296] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:21:55 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:21:55.955785 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fad9e42-c6a0-40bf-a41f-689c6bd1d266 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:21:55.972542 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fad9e42-c6a0-40bf-a41f-689c6bd1d266 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:21:55.979055 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73184360-c5fc-49d0-a7ae-b82d06ee6325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:21:55.979701 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73184360-c5fc-49d0-a7ae-b82d06ee6325 admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:21:55.979977 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73184360-c5fc-49d0-a7ae-b82d06ee6325 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:21:55.980254 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73184360-c5fc-49d0-a7ae-b82d06ee6325 admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:21:55.981416 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73184360-c5fc-49d0-a7ae-b82d06ee6325 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:21:55.983164 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73184360-c5fc-49d0-a7ae-b82d06ee6325 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:21:56.005183 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1139/2297] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:21:55 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:21:56.010386 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdde7075-6cd4-4876-8f76-e0f312b9df3f admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:21:56.030001 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdde7075-6cd4-4876-8f76-e0f312b9df3f admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:21:56.038308 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddae28b-3147-4dfc-9025-8be2ecb94f7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:21:56.039459 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddae28b-3147-4dfc-9025-8be2ecb94f7e admin admin] REQUEST_METHOD: `PUT` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:21:56.039950 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddae28b-3147-4dfc-9025-8be2ecb94f7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:21:56.040469 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddae28b-3147-4dfc-9025-8be2ecb94f7e admin admin] PATH_INFO: `/v3/projects/687aee32ec044228850767897e714c2f/users/a2fe1244fcbd44e1ab7d8f89d5d1ae0f/roles/4eea39e24b06431aa357053fe7907a3d` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:21:56.069864 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddae28b-3147-4dfc-9025-8be2ecb94f7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=687aee32ec044228850767897e714c2f, user_id=a2fe1244fcbd44e1ab7d8f89d5d1ae0f, role_id=4eea39e24b06431aa357053fe7907a3d)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:21:56.075494 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddae28b-3147-4dfc-9025-8be2ecb94f7e admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:21:56.097199 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1159/2298] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:21:56 2022] PUT /identity/v3/projects/687aee32ec044228850767897e714c2f/users/a2fe1244fcbd44e1ab7d8f89d5d1ae0f/roles/4eea39e24b06431aa357053fe7907a3d => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:21:56.108698 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efea480a-568b-4f2b-b620-97a2d85bae33 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:21:56.109034 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efea480a-568b-4f2b-b620-97a2d85bae33 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:21:56.109339 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efea480a-568b-4f2b-b620-97a2d85bae33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:21:56.145222 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-efea480a-568b-4f2b-b620-97a2d85bae33 None None] MFA Rules not processed for user `a2fe1244fcbd44e1ab7d8f89d5d1ae0f`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:21:56.146824 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-efea480a-568b-4f2b-b620-97a2d85bae33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:21:56.261535 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1140/2299] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:21:56 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:21:56.264794 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddae28b-3147-4dfc-9025-8be2ecb94f7e admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:21:56.370770 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddae28b-3147-4dfc-9025-8be2ecb94f7e admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:21:56.379298 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02b5680-b968-4bec-9bdf-20f283aa9a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:21:56.380182 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02b5680-b968-4bec-9bdf-20f283aa9a4d admin admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:21:56.380622 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02b5680-b968-4bec-9bdf-20f283aa9a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:21:56.381076 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02b5680-b968-4bec-9bdf-20f283aa9a4d admin admin] PATH_INFO: `/v3/projects` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:21:56.382217 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02b5680-b968-4bec-9bdf-20f283aa9a4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:21:56.385584 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02b5680-b968-4bec-9bdf-20f283aa9a4d admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:21:56.428147 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1160/2300] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:21:56 2022] POST /identity/v3/projects => generated 387 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:21:56.429776 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efea480a-568b-4f2b-b620-97a2d85bae33 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:21:56.443491 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efea480a-568b-4f2b-b620-97a2d85bae33 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:21:56.465729 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:21:56.563671 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2568a2a5-e38d-4493-aab3-1123d2092c45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:21:56.564863 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2568a2a5-e38d-4493-aab3-1123d2092c45 admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:21:56.565366 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2568a2a5-e38d-4493-aab3-1123d2092c45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:21:56.566587 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2568a2a5-e38d-4493-aab3-1123d2092c45 admin admin] PATH_INFO: `/v3/users` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:21:56.567151 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2568a2a5-e38d-4493-aab3-1123d2092c45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:21:56.571341 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2568a2a5-e38d-4493-aab3-1123d2092c45 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:21:56.614599 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1141/2301] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:21:56 2022] POST /identity/v3/users => generated 300 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:21:56.619948 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02b5680-b968-4bec-9bdf-20f283aa9a4d admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:21:56.648472 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02b5680-b968-4bec-9bdf-20f283aa9a4d admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:21:56.663116 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572e5897-7f93-4f3c-81ca-8ae6d1bc9dec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:21:56.664026 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572e5897-7f93-4f3c-81ca-8ae6d1bc9dec admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:21:56.664629 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572e5897-7f93-4f3c-81ca-8ae6d1bc9dec admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:21:56.665187 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572e5897-7f93-4f3c-81ca-8ae6d1bc9dec admin admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:21:56.666500 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572e5897-7f93-4f3c-81ca-8ae6d1bc9dec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:21:56.670676 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572e5897-7f93-4f3c-81ca-8ae6d1bc9dec admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:21:56.692728 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1161/2302] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:21:56 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:21:56.698089 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2568a2a5-e38d-4493-aab3-1123d2092c45 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:21:56.723791 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2568a2a5-e38d-4493-aab3-1123d2092c45 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:21:56.735545 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a3bdfd-b4a8-44ab-87c7-31d901fe0dec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:21:56.738645 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a3bdfd-b4a8-44ab-87c7-31d901fe0dec admin admin] REQUEST_METHOD: `PUT` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:21:56.738645 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a3bdfd-b4a8-44ab-87c7-31d901fe0dec admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:21:56.738645 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a3bdfd-b4a8-44ab-87c7-31d901fe0dec admin admin] PATH_INFO: `/v3/projects/4f1b71e835b44c9ca91fa46e6736e642/users/82bf7e67cfeb46c1b099f38f0633e194/roles/e0ab41d8ae22474688b5361bcc75e0f0` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:21:56.771779 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a3bdfd-b4a8-44ab-87c7-31d901fe0dec admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f1b71e835b44c9ca91fa46e6736e642, user_id=82bf7e67cfeb46c1b099f38f0633e194, role_id=e0ab41d8ae22474688b5361bcc75e0f0)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:21:56.776134 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a3bdfd-b4a8-44ab-87c7-31d901fe0dec admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:21:56.809958 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1142/2303] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:21:56 2022] PUT /identity/v3/projects/4f1b71e835b44c9ca91fa46e6736e642/users/82bf7e67cfeb46c1b099f38f0633e194/roles/e0ab41d8ae22474688b5361bcc75e0f0 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:21:56.815047 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572e5897-7f93-4f3c-81ca-8ae6d1bc9dec admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:21:56.836963 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572e5897-7f93-4f3c-81ca-8ae6d1bc9dec admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:21:56.874829 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:21:56.943929 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76d2889-8259-4391-9d57-e7d9a00977ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:21:56.944892 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76d2889-8259-4391-9d57-e7d9a00977ae admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:21:56.945565 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76d2889-8259-4391-9d57-e7d9a00977ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:21:56.946044 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76d2889-8259-4391-9d57-e7d9a00977ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:21:56.947263 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76d2889-8259-4391-9d57-e7d9a00977ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:21:56.949439 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76d2889-8259-4391-9d57-e7d9a00977ae admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:21:56.964422 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1162/2304] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:21:56 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:21:56.973426 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a3bdfd-b4a8-44ab-87c7-31d901fe0dec admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:21:57.002795 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a3bdfd-b4a8-44ab-87c7-31d901fe0dec admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:21:57.015100 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0244d5b-6324-4bb1-924c-1cbef61c0f86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:21:57.016299 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0244d5b-6324-4bb1-924c-1cbef61c0f86 admin admin] REQUEST_METHOD: `PUT` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:21:57.016855 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0244d5b-6324-4bb1-924c-1cbef61c0f86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:21:57.017365 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0244d5b-6324-4bb1-924c-1cbef61c0f86 admin admin] PATH_INFO: `/v3/projects/4f1b71e835b44c9ca91fa46e6736e642/users/82bf7e67cfeb46c1b099f38f0633e194/roles/e0ab41d8ae22474688b5361bcc75e0f0` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:21:57.022400 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0244d5b-6324-4bb1-924c-1cbef61c0f86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f1b71e835b44c9ca91fa46e6736e642, user_id=82bf7e67cfeb46c1b099f38f0633e194, role_id=e0ab41d8ae22474688b5361bcc75e0f0)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:21:57.026142 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0244d5b-6324-4bb1-924c-1cbef61c0f86 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:21:57.042327 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1143/2305] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:21:56 2022] PUT /identity/v3/projects/4f1b71e835b44c9ca91fa46e6736e642/users/82bf7e67cfeb46c1b099f38f0633e194/roles/e0ab41d8ae22474688b5361bcc75e0f0 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:21:57.062711 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa1075f-c5f9-4786-81a3-46dae2bddd65 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:21:57.063124 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa1075f-c5f9-4786-81a3-46dae2bddd65 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:21:57.063529 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa1075f-c5f9-4786-81a3-46dae2bddd65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:21:57.106856 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-baa1075f-c5f9-4786-81a3-46dae2bddd65 None None] MFA Rules not processed for user `82bf7e67cfeb46c1b099f38f0633e194`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:21:57.108515 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-baa1075f-c5f9-4786-81a3-46dae2bddd65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:21:57.224852 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1163/2306] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:21:57 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:21:57.243355 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0244d5b-6324-4bb1-924c-1cbef61c0f86 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:21:57.245750 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-f0244d5b-6324-4bb1-924c-1cbef61c0f86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:21:57.326111 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0244d5b-6324-4bb1-924c-1cbef61c0f86 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:21:57.330617 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36dd49c-c8fa-4d5a-b212-c153d7edb5f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df1bca993bc142908cc7cf85eabd2b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:21:57.331302 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36dd49c-c8fa-4d5a-b212-c153d7edb5f7 service cinder] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:21:57.331721 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36dd49c-c8fa-4d5a-b212-c153d7edb5f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:21:57.332070 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36dd49c-c8fa-4d5a-b212-c153d7edb5f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:21:57.343376 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36dd49c-c8fa-4d5a-b212-c153d7edb5f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:21:57.344957 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36dd49c-c8fa-4d5a-b212-c153d7edb5f7 service cinder] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:21:57.357577 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1144/2307] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:21:57 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 20:21:57.411953 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa1075f-c5f9-4786-81a3-46dae2bddd65 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:21:57.503054 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa1075f-c5f9-4786-81a3-46dae2bddd65 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:21:57.508384 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aa019e-664d-4a65-959c-632c6aee0912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:21:57.509157 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aa019e-664d-4a65-959c-632c6aee0912 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:21:57.509487 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aa019e-664d-4a65-959c-632c6aee0912 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:21:57.509897 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aa019e-664d-4a65-959c-632c6aee0912 admin admin] PATH_INFO: `/v3/users/a2fe1244fcbd44e1ab7d8f89d5d1ae0f` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:21:57.512466 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aa019e-664d-4a65-959c-632c6aee0912 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2fe1244fcbd44e1ab7d8f89d5d1ae0f)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:21:57.514278 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aa019e-664d-4a65-959c-632c6aee0912 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:21:57.578374 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-76aa019e-664d-4a65-959c-632c6aee0912 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2fe1244fcbd44e1ab7d8f89d5d1ae0f'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:21:57.579094 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-76aa019e-664d-4a65-959c-632c6aee0912 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2fe1244fcbd44e1ab7d8f89d5d1ae0f'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:21:57.582093 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:21:57.598083 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-76aa019e-664d-4a65-959c-632c6aee0912 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2fe1244fcbd44e1ab7d8f89d5d1ae0f'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:21:57.599243 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-76aa019e-664d-4a65-959c-632c6aee0912 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2fe1244fcbd44e1ab7d8f89d5d1ae0f'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:21:57.610107 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-76aa019e-664d-4a65-959c-632c6aee0912 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2fe1244fcbd44e1ab7d8f89d5d1ae0f'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:21:57.630530 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1164/2308] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:21:57 2022] DELETE /identity/v3/users/a2fe1244fcbd44e1ab7d8f89d5d1ae0f => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:21:57.638307 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d4e116-f1f3-43b9-8cf5-9b3231e982d4 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:21:57.639064 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d4e116-f1f3-43b9-8cf5-9b3231e982d4 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:21:57.639660 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d4e116-f1f3-43b9-8cf5-9b3231e982d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:21:57.668376 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-b4d4e116-f1f3-43b9-8cf5-9b3231e982d4 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:21:57.670857 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-b4d4e116-f1f3-43b9-8cf5-9b3231e982d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:21:57.749986 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1145/2309] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:21:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:21:57.764921 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aa019e-664d-4a65-959c-632c6aee0912 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:21:57.766921 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-76aa019e-664d-4a65-959c-632c6aee0912 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:21:57.866354 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aa019e-664d-4a65-959c-632c6aee0912 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:21:57.873072 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9a45dc-f204-4bd4-bb83-fd95c0a24700 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:21:57.873883 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9a45dc-f204-4bd4-bb83-fd95c0a24700 service neutron] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:21:57.874170 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9a45dc-f204-4bd4-bb83-fd95c0a24700 service neutron] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:21:57.874450 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9a45dc-f204-4bd4-bb83-fd95c0a24700 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:21:57.891958 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9a45dc-f204-4bd4-bb83-fd95c0a24700 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:21:57.895062 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9a45dc-f204-4bd4-bb83-fd95c0a24700 service neutron] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:21:57.917555 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1165/2310] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:21:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:21:58.363747 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d4e116-f1f3-43b9-8cf5-9b3231e982d4 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:21:58.366883 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-b4d4e116-f1f3-43b9-8cf5-9b3231e982d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:21:58.381339 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d4e116-f1f3-43b9-8cf5-9b3231e982d4 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:21:58.386909 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9b421a-7e2c-4ced-9a9a-f23847ca7e9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:21:58.387684 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9b421a-7e2c-4ced-9a9a-f23847ca7e9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:21:58.388012 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9b421a-7e2c-4ced-9a9a-f23847ca7e9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:21:58.388391 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9b421a-7e2c-4ced-9a9a-f23847ca7e9e admin admin] PATH_INFO: `/v3/projects/687aee32ec044228850767897e714c2f` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:21:58.389873 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9b421a-7e2c-4ced-9a9a-f23847ca7e9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=687aee32ec044228850767897e714c2f)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:21:58.392157 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9b421a-7e2c-4ced-9a9a-f23847ca7e9e admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:21:58.400985 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:21:58.470329 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-4b9b421a-7e2c-4ced-9a9a-f23847ca7e9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '687aee32ec044228850767897e714c2f', 'request_id': 'req-4b9b421a-7e2c-4ced-9a9a-f23847ca7e9e'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:21:58.479270 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-4b9b421a-7e2c-4ced-9a9a-f23847ca7e9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '687aee32ec044228850767897e714c2f', 'request_id': 'req-4b9b421a-7e2c-4ced-9a9a-f23847ca7e9e'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:21:58.489993 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-4b9b421a-7e2c-4ced-9a9a-f23847ca7e9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '687aee32ec044228850767897e714c2f', 'request_id': 'req-4b9b421a-7e2c-4ced-9a9a-f23847ca7e9e'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:21:58.513497 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-4b9b421a-7e2c-4ced-9a9a-f23847ca7e9e admin admin] The token cache is being invalidate because project 687aee32ec044228850767897e714c2f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81771) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:21:58.514168 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-4b9b421a-7e2c-4ced-9a9a-f23847ca7e9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:21:58.515250 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-4b9b421a-7e2c-4ced-9a9a-f23847ca7e9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:21:58.517581 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1146/2311] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:21:58 2022] DELETE /identity/v3/projects/687aee32ec044228850767897e714c2f => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:21:58.523986 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9a45dc-f204-4bd4-bb83-fd95c0a24700 service neutron] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:21:58.547754 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9a45dc-f204-4bd4-bb83-fd95c0a24700 service neutron] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:21:58.651203 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8175f32e-7549-4062-8bed-3495e65247ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:21:58.652761 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8175f32e-7549-4062-8bed-3495e65247ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:21:58.653263 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8175f32e-7549-4062-8bed-3495e65247ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:21:58.653922 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8175f32e-7549-4062-8bed-3495e65247ea admin admin] PATH_INFO: `/v3/users/82bf7e67cfeb46c1b099f38f0633e194` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:21:58.656265 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8175f32e-7549-4062-8bed-3495e65247ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=82bf7e67cfeb46c1b099f38f0633e194)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:21:58.660280 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8175f32e-7549-4062-8bed-3495e65247ea admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:21:58.736220 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-8175f32e-7549-4062-8bed-3495e65247ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82bf7e67cfeb46c1b099f38f0633e194'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:21:58.737202 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-8175f32e-7549-4062-8bed-3495e65247ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82bf7e67cfeb46c1b099f38f0633e194'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:21:58.753060 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-8175f32e-7549-4062-8bed-3495e65247ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82bf7e67cfeb46c1b099f38f0633e194'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:21:58.753923 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-8175f32e-7549-4062-8bed-3495e65247ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82bf7e67cfeb46c1b099f38f0633e194'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:21:58.764370 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-8175f32e-7549-4062-8bed-3495e65247ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82bf7e67cfeb46c1b099f38f0633e194'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:21:58.766446 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:21:58.786542 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1166/2312] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:21:58 2022] DELETE /identity/v3/users/82bf7e67cfeb46c1b099f38f0633e194 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:21:59.193390 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9b421a-7e2c-4ced-9a9a-f23847ca7e9e admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:21:59.195722 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-4b9b421a-7e2c-4ced-9a9a-f23847ca7e9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:21:59.221765 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9b421a-7e2c-4ced-9a9a-f23847ca7e9e admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:21:59.337158 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7858e8dc-a537-444b-80e3-065003ee52e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:21:59.337867 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7858e8dc-a537-444b-80e3-065003ee52e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:21:59.338251 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7858e8dc-a537-444b-80e3-065003ee52e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:21:59.338727 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7858e8dc-a537-444b-80e3-065003ee52e9 admin admin] PATH_INFO: `/v3/projects/4f1b71e835b44c9ca91fa46e6736e642` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:21:59.341328 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7858e8dc-a537-444b-80e3-065003ee52e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f1b71e835b44c9ca91fa46e6736e642)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:21:59.343581 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7858e8dc-a537-444b-80e3-065003ee52e9 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:21:59.416632 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-7858e8dc-a537-444b-80e3-065003ee52e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f1b71e835b44c9ca91fa46e6736e642', 'request_id': 'req-7858e8dc-a537-444b-80e3-065003ee52e9'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:21:59.430059 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-7858e8dc-a537-444b-80e3-065003ee52e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f1b71e835b44c9ca91fa46e6736e642', 'request_id': 'req-7858e8dc-a537-444b-80e3-065003ee52e9'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:21:59.444536 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-7858e8dc-a537-444b-80e3-065003ee52e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f1b71e835b44c9ca91fa46e6736e642', 'request_id': 'req-7858e8dc-a537-444b-80e3-065003ee52e9'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:21:59.467192 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-7858e8dc-a537-444b-80e3-065003ee52e9 admin admin] The token cache is being invalidate because project 4f1b71e835b44c9ca91fa46e6736e642 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81771) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:21:59.467507 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-7858e8dc-a537-444b-80e3-065003ee52e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:21:59.470753 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-7858e8dc-a537-444b-80e3-065003ee52e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:21:59.472552 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1147/2313] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:21:59 2022] DELETE /identity/v3/projects/4f1b71e835b44c9ca91fa46e6736e642 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:21:59.480057 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6b0d3e-0aac-4e31-84f2-4811637427e1 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:21:59.480941 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6b0d3e-0aac-4e31-84f2-4811637427e1 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:21:59.481234 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6b0d3e-0aac-4e31-84f2-4811637427e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:21:59.507549 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-ff6b0d3e-0aac-4e31-84f2-4811637427e1 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:21:59.508877 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-ff6b0d3e-0aac-4e31-84f2-4811637427e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:21:59.590608 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1167/2314] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:21:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:21:59.600295 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f3a4d2-d762-4818-ade0-fd9f157dd8e2 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:21:59.600682 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f3a4d2-d762-4818-ade0-fd9f157dd8e2 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:21:59.600878 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f3a4d2-d762-4818-ade0-fd9f157dd8e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:21:59.622745 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-51f3a4d2-d762-4818-ade0-fd9f157dd8e2 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:21:59.624015 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-51f3a4d2-d762-4818-ade0-fd9f157dd8e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:21:59.633011 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1148/2315] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:21:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:21:59.638961 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6b0d3e-0aac-4e31-84f2-4811637427e1 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:21:59.657997 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6b0d3e-0aac-4e31-84f2-4811637427e1 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:21:59.663035 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14387bab-218d-4d2f-a2bc-bd2ba217264e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:21:59.663694 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14387bab-218d-4d2f-a2bc-bd2ba217264e admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:21:59.663983 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14387bab-218d-4d2f-a2bc-bd2ba217264e admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:21:59.664281 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14387bab-218d-4d2f-a2bc-bd2ba217264e admin admin] PATH_INFO: `/v3/domains` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:21:59.664951 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14387bab-218d-4d2f-a2bc-bd2ba217264e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:21:59.665257 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14387bab-218d-4d2f-a2bc-bd2ba217264e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:21:59.667011 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14387bab-218d-4d2f-a2bc-bd2ba217264e admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:21:59.685677 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1168/2316] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 6 20:21:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:21:59.689395 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f3a4d2-d762-4818-ade0-fd9f157dd8e2 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:21:59.701288 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f3a4d2-d762-4818-ade0-fd9f157dd8e2 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:21:59.706877 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfe7951-417a-4e25-beea-ef5c373e13d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:21:59.707501 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfe7951-417a-4e25-beea-ef5c373e13d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:21:59.707780 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfe7951-417a-4e25-beea-ef5c373e13d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:21:59.708055 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfe7951-417a-4e25-beea-ef5c373e13d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:21:59.708734 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfe7951-417a-4e25-beea-ef5c373e13d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:21:59.710925 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfe7951-417a-4e25-beea-ef5c373e13d0 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:21:59.731151 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:21:59.749307 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1149/2317] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:21:59 2022] POST /identity/v3/projects => generated 389 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 20:21:59.754661 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14387bab-218d-4d2f-a2bc-bd2ba217264e admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:21:59.766896 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14387bab-218d-4d2f-a2bc-bd2ba217264e admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:21:59.842134 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315a5774-4811-41f6-a3e0-2a7ed01d2dad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:21:59.842769 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315a5774-4811-41f6-a3e0-2a7ed01d2dad admin admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:21:59.843050 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315a5774-4811-41f6-a3e0-2a7ed01d2dad admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:21:59.843328 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315a5774-4811-41f6-a3e0-2a7ed01d2dad admin admin] PATH_INFO: `/v3/users` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:21:59.843981 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315a5774-4811-41f6-a3e0-2a7ed01d2dad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:21:59.846037 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315a5774-4811-41f6-a3e0-2a7ed01d2dad admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:21:59.871512 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1169/2318] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:21:59 2022] POST /identity/v3/users => generated 295 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:21:59.876574 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfe7951-417a-4e25-beea-ef5c373e13d0 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:21:59.895644 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfe7951-417a-4e25-beea-ef5c373e13d0 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:21:59.904813 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b22637-5f1d-410c-a304-bc894ff93c8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:21:59.905935 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b22637-5f1d-410c-a304-bc894ff93c8b admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:21:59.906448 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b22637-5f1d-410c-a304-bc894ff93c8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:21:59.906998 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b22637-5f1d-410c-a304-bc894ff93c8b admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:21:59.908222 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b22637-5f1d-410c-a304-bc894ff93c8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:21:59.913822 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b22637-5f1d-410c-a304-bc894ff93c8b admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:21:59.932881 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1150/2319] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:21:59 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:21:59.938051 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315a5774-4811-41f6-a3e0-2a7ed01d2dad admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:21:59.951509 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315a5774-4811-41f6-a3e0-2a7ed01d2dad admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:21:59.956876 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf21111e-7588-47fe-8c33-2730d6673826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:21:59.957512 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf21111e-7588-47fe-8c33-2730d6673826 admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:21:59.957821 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf21111e-7588-47fe-8c33-2730d6673826 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:21:59.958104 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf21111e-7588-47fe-8c33-2730d6673826 admin admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:21:59.958774 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf21111e-7588-47fe-8c33-2730d6673826 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:21:59.960493 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf21111e-7588-47fe-8c33-2730d6673826 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:21:59.973596 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1170/2320] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:21:59 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 20:21:59.978733 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b22637-5f1d-410c-a304-bc894ff93c8b admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:21:59.997833 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b22637-5f1d-410c-a304-bc894ff93c8b admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:22:00.006067 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129bd414-109a-4fba-96b5-6d243f805d7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:22:00.007212 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129bd414-109a-4fba-96b5-6d243f805d7b admin admin] REQUEST_METHOD: `PUT` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:22:00.007698 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129bd414-109a-4fba-96b5-6d243f805d7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:22:00.008179 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129bd414-109a-4fba-96b5-6d243f805d7b admin admin] PATH_INFO: `/v3/projects/13b131d6231b449a851ece2e4c8e5f88/users/7fcfd354986040a5a06c5ee301506485/roles/4eea39e24b06431aa357053fe7907a3d` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:22:00.043969 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129bd414-109a-4fba-96b5-6d243f805d7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=13b131d6231b449a851ece2e4c8e5f88, user_id=7fcfd354986040a5a06c5ee301506485, role_id=4eea39e24b06431aa357053fe7907a3d)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:22:00.049927 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129bd414-109a-4fba-96b5-6d243f805d7b admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:22:00.070559 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1151/2321] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:21:59 2022] PUT /identity/v3/projects/13b131d6231b449a851ece2e4c8e5f88/users/7fcfd354986040a5a06c5ee301506485/roles/4eea39e24b06431aa357053fe7907a3d => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:22:00.082231 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbd5526-2c03-4e36-a8e0-720930aae422 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:22:00.082595 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbd5526-2c03-4e36-a8e0-720930aae422 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:22:00.083060 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbd5526-2c03-4e36-a8e0-720930aae422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:22:00.119603 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-2bbd5526-2c03-4e36-a8e0-720930aae422 None None] MFA Rules not processed for user `7fcfd354986040a5a06c5ee301506485`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:22:00.121441 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-2bbd5526-2c03-4e36-a8e0-720930aae422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:22:00.183250 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:22:00.221890 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1171/2322] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:22:00 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:22:00.229219 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129bd414-109a-4fba-96b5-6d243f805d7b admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:22:00.312065 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129bd414-109a-4fba-96b5-6d243f805d7b admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:22:00.318407 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077ca31c-f81c-4460-9d61-83371a689cf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:22:00.319437 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077ca31c-f81c-4460-9d61-83371a689cf3 admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:22:00.319816 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077ca31c-f81c-4460-9d61-83371a689cf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:22:00.320283 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077ca31c-f81c-4460-9d61-83371a689cf3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:22:00.321359 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077ca31c-f81c-4460-9d61-83371a689cf3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:22:00.323447 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077ca31c-f81c-4460-9d61-83371a689cf3 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:22:00.354171 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1152/2323] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:22:00 2022] POST /identity/v3/projects => generated 387 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 20:22:00.359227 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbd5526-2c03-4e36-a8e0-720930aae422 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:22:00.371594 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbd5526-2c03-4e36-a8e0-720930aae422 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:22:00.459011 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bc36c8-3ea6-4b93-9007-ab227f332cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:22:00.459737 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bc36c8-3ea6-4b93-9007-ab227f332cb5 admin admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:22:00.460236 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bc36c8-3ea6-4b93-9007-ab227f332cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:22:00.460729 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bc36c8-3ea6-4b93-9007-ab227f332cb5 admin admin] PATH_INFO: `/v3/users` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:22:00.462014 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bc36c8-3ea6-4b93-9007-ab227f332cb5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:22:00.465891 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bc36c8-3ea6-4b93-9007-ab227f332cb5 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:22:00.502957 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1172/2324] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:22:00 2022] POST /identity/v3/users => generated 300 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 20:22:00.504534 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077ca31c-f81c-4460-9d61-83371a689cf3 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:22:00.516398 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077ca31c-f81c-4460-9d61-83371a689cf3 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:22:00.521733 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76c5028-24e4-4f4f-9a2e-347d1f17997a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:22:00.522375 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76c5028-24e4-4f4f-9a2e-347d1f17997a admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:22:00.522748 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76c5028-24e4-4f4f-9a2e-347d1f17997a admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:22:00.523050 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76c5028-24e4-4f4f-9a2e-347d1f17997a admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:22:00.523680 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76c5028-24e4-4f4f-9a2e-347d1f17997a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:22:00.525331 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76c5028-24e4-4f4f-9a2e-347d1f17997a admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:22:00.537857 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1153/2325] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:22:00 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:22:00.542413 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bc36c8-3ea6-4b93-9007-ab227f332cb5 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:22:00.553834 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bc36c8-3ea6-4b93-9007-ab227f332cb5 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:22:00.559171 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c977687-ca54-4c17-ae50-a7162274c5ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:22:00.559861 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c977687-ca54-4c17-ae50-a7162274c5ef admin admin] REQUEST_METHOD: `PUT` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:22:00.560189 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c977687-ca54-4c17-ae50-a7162274c5ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:22:00.560474 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c977687-ca54-4c17-ae50-a7162274c5ef admin admin] PATH_INFO: `/v3/projects/b13388d4868f4e20864737abe9dbfbf8/users/216bcee1c0264184b196514311b8ce9d/roles/e0ab41d8ae22474688b5361bcc75e0f0` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:22:00.577549 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c977687-ca54-4c17-ae50-a7162274c5ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=b13388d4868f4e20864737abe9dbfbf8, user_id=216bcee1c0264184b196514311b8ce9d, role_id=e0ab41d8ae22474688b5361bcc75e0f0)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:22:00.580294 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c977687-ca54-4c17-ae50-a7162274c5ef admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:22:00.595588 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1173/2326] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:22:00 2022] PUT /identity/v3/projects/b13388d4868f4e20864737abe9dbfbf8/users/216bcee1c0264184b196514311b8ce9d/roles/e0ab41d8ae22474688b5361bcc75e0f0 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:22:00.600216 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76c5028-24e4-4f4f-9a2e-347d1f17997a admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:22:00.611145 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76c5028-24e4-4f4f-9a2e-347d1f17997a admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:22:00.677444 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae872c60-c0e6-4a56-b108-f2a7d1f2a234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:22:00.678116 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae872c60-c0e6-4a56-b108-f2a7d1f2a234 admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:22:00.678405 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae872c60-c0e6-4a56-b108-f2a7d1f2a234 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:22:00.678687 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae872c60-c0e6-4a56-b108-f2a7d1f2a234 admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:22:00.679357 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae872c60-c0e6-4a56-b108-f2a7d1f2a234 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:22:00.681005 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae872c60-c0e6-4a56-b108-f2a7d1f2a234 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:22:00.693503 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1154/2327] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:22:00 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:22:00.698633 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c977687-ca54-4c17-ae50-a7162274c5ef admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:22:00.717917 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c977687-ca54-4c17-ae50-a7162274c5ef admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:22:00.726236 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1908a1-1008-4245-b03a-0673df609238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:22:00.727300 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1908a1-1008-4245-b03a-0673df609238 admin admin] REQUEST_METHOD: `PUT` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:22:00.727688 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1908a1-1008-4245-b03a-0673df609238 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:22:00.728063 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1908a1-1008-4245-b03a-0673df609238 admin admin] PATH_INFO: `/v3/projects/b13388d4868f4e20864737abe9dbfbf8/users/216bcee1c0264184b196514311b8ce9d/roles/e0ab41d8ae22474688b5361bcc75e0f0` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:22:00.731774 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1908a1-1008-4245-b03a-0673df609238 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b13388d4868f4e20864737abe9dbfbf8, user_id=216bcee1c0264184b196514311b8ce9d, role_id=e0ab41d8ae22474688b5361bcc75e0f0)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:22:00.737305 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1908a1-1008-4245-b03a-0673df609238 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:22:00.757934 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1174/2328] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:22:00 2022] PUT /identity/v3/projects/b13388d4868f4e20864737abe9dbfbf8/users/216bcee1c0264184b196514311b8ce9d/roles/e0ab41d8ae22474688b5361bcc75e0f0 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:22:00.776124 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15fce8a-1d17-45bc-90d2-0c11acdaaba1 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:22:00.776572 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15fce8a-1d17-45bc-90d2-0c11acdaaba1 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:22:00.777715 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15fce8a-1d17-45bc-90d2-0c11acdaaba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:22:00.820316 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-d15fce8a-1d17-45bc-90d2-0c11acdaaba1 None None] MFA Rules not processed for user `216bcee1c0264184b196514311b8ce9d`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:22:00.822097 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-d15fce8a-1d17-45bc-90d2-0c11acdaaba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:22:00.846943 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:22:00.953518 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1155/2329] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:22:00 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:22:00.965883 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1908a1-1008-4245-b03a-0673df609238 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:22:00.967975 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-6b1908a1-1008-4245-b03a-0673df609238 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:22:01.066532 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1908a1-1008-4245-b03a-0673df609238 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:22:01.071107 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219df169-b7fe-4dc9-953e-14aed1df87d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df1bca993bc142908cc7cf85eabd2b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:22:01.071745 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219df169-b7fe-4dc9-953e-14aed1df87d8 service cinder] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:22:01.072029 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219df169-b7fe-4dc9-953e-14aed1df87d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:22:01.072336 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219df169-b7fe-4dc9-953e-14aed1df87d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:22:01.084972 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219df169-b7fe-4dc9-953e-14aed1df87d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:22:01.086589 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219df169-b7fe-4dc9-953e-14aed1df87d8 service cinder] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:22:01.098184 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1175/2330] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:22:00 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:22:01.267601 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15fce8a-1d17-45bc-90d2-0c11acdaaba1 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:22:01.369248 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15fce8a-1d17-45bc-90d2-0c11acdaaba1 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:22:01.377916 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9572e78-3c25-43f1-aa47-c05c48414f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:22:01.379040 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9572e78-3c25-43f1-aa47-c05c48414f48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:22:01.379632 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9572e78-3c25-43f1-aa47-c05c48414f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:22:01.380067 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9572e78-3c25-43f1-aa47-c05c48414f48 admin admin] PATH_INFO: `/v3/users/7fcfd354986040a5a06c5ee301506485` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:22:01.382276 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9572e78-3c25-43f1-aa47-c05c48414f48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7fcfd354986040a5a06c5ee301506485)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:22:01.386242 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9572e78-3c25-43f1-aa47-c05c48414f48 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:22:01.462792 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-b9572e78-3c25-43f1-aa47-c05c48414f48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7fcfd354986040a5a06c5ee301506485'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:22:01.474949 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-b9572e78-3c25-43f1-aa47-c05c48414f48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7fcfd354986040a5a06c5ee301506485'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:22:01.475963 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-b9572e78-3c25-43f1-aa47-c05c48414f48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7fcfd354986040a5a06c5ee301506485'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:22:01.476944 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-b9572e78-3c25-43f1-aa47-c05c48414f48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7fcfd354986040a5a06c5ee301506485'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:22:01.490842 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-b9572e78-3c25-43f1-aa47-c05c48414f48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7fcfd354986040a5a06c5ee301506485'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:22:01.523506 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1156/2331] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:22:01 2022] DELETE /identity/v3/users/7fcfd354986040a5a06c5ee301506485 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:22:01.531519 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dc6915-4657-43f6-ae61-b33be98e41f9 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:22:01.532080 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dc6915-4657-43f6-ae61-b33be98e41f9 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:22:01.532578 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dc6915-4657-43f6-ae61-b33be98e41f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:22:01.564543 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-56dc6915-4657-43f6-ae61-b33be98e41f9 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:22:01.566593 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-56dc6915-4657-43f6-ae61-b33be98e41f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:22:01.609006 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:22:01.653819 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1176/2332] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:22:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:22:01.668487 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9572e78-3c25-43f1-aa47-c05c48414f48 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:22:01.670764 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-b9572e78-3c25-43f1-aa47-c05c48414f48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:22:01.745545 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:22:01.777997 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9572e78-3c25-43f1-aa47-c05c48414f48 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:22:01.783499 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b305e6-164d-49f1-b0ed-75b23f22f32b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:22:01.783499 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b305e6-164d-49f1-b0ed-75b23f22f32b service neutron] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:22:01.784279 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b305e6-164d-49f1-b0ed-75b23f22f32b service neutron] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:22:01.784279 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b305e6-164d-49f1-b0ed-75b23f22f32b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:22:01.798144 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b305e6-164d-49f1-b0ed-75b23f22f32b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:22:01.799741 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b305e6-164d-49f1-b0ed-75b23f22f32b service neutron] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:22:01.812488 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1157/2333] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:22:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:22:02.310675 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dc6915-4657-43f6-ae61-b33be98e41f9 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:22:02.312339 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-56dc6915-4657-43f6-ae61-b33be98e41f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:22:02.336145 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dc6915-4657-43f6-ae61-b33be98e41f9 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:22:02.345002 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c74573-130a-42b1-b3e5-1a461345e41d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:22:02.345777 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c74573-130a-42b1-b3e5-1a461345e41d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:22:02.346125 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c74573-130a-42b1-b3e5-1a461345e41d admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:22:02.346403 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c74573-130a-42b1-b3e5-1a461345e41d admin admin] PATH_INFO: `/v3/projects/13b131d6231b449a851ece2e4c8e5f88` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:22:02.348574 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c74573-130a-42b1-b3e5-1a461345e41d admin admin] RBAC: Authorizing `identity:delete_project(project_id=13b131d6231b449a851ece2e4c8e5f88)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:22:02.352749 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c74573-130a-42b1-b3e5-1a461345e41d admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:22:02.458207 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-d7c74573-130a-42b1-b3e5-1a461345e41d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13b131d6231b449a851ece2e4c8e5f88', 'request_id': 'req-d7c74573-130a-42b1-b3e5-1a461345e41d'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:22:02.469519 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-d7c74573-130a-42b1-b3e5-1a461345e41d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13b131d6231b449a851ece2e4c8e5f88', 'request_id': 'req-d7c74573-130a-42b1-b3e5-1a461345e41d'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:22:02.477935 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-d7c74573-130a-42b1-b3e5-1a461345e41d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13b131d6231b449a851ece2e4c8e5f88', 'request_id': 'req-d7c74573-130a-42b1-b3e5-1a461345e41d'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:22:02.500804 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-d7c74573-130a-42b1-b3e5-1a461345e41d admin admin] The token cache is being invalidate because project 13b131d6231b449a851ece2e4c8e5f88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81772) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:22:02.501317 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-d7c74573-130a-42b1-b3e5-1a461345e41d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:22:02.503647 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-d7c74573-130a-42b1-b3e5-1a461345e41d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:22:02.509749 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1177/2334] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:22:02 2022] DELETE /identity/v3/projects/13b131d6231b449a851ece2e4c8e5f88 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 20:22:02.510975 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b305e6-164d-49f1-b0ed-75b23f22f32b service neutron] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:22:02.522612 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b305e6-164d-49f1-b0ed-75b23f22f32b service neutron] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:22:02.598545 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccfb56f-4bbc-4858-9391-d12a1e32d8d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:22:02.599379 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccfb56f-4bbc-4858-9391-d12a1e32d8d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:22:02.599696 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccfb56f-4bbc-4858-9391-d12a1e32d8d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:22:02.599968 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccfb56f-4bbc-4858-9391-d12a1e32d8d7 admin admin] PATH_INFO: `/v3/users/216bcee1c0264184b196514311b8ce9d` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:22:02.601534 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccfb56f-4bbc-4858-9391-d12a1e32d8d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=216bcee1c0264184b196514311b8ce9d)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:22:02.603557 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccfb56f-4bbc-4858-9391-d12a1e32d8d7 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:22:02.672615 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-bccfb56f-4bbc-4858-9391-d12a1e32d8d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '216bcee1c0264184b196514311b8ce9d'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:22:02.682698 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-bccfb56f-4bbc-4858-9391-d12a1e32d8d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '216bcee1c0264184b196514311b8ce9d'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:22:02.683896 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-bccfb56f-4bbc-4858-9391-d12a1e32d8d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '216bcee1c0264184b196514311b8ce9d'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:22:02.684655 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-bccfb56f-4bbc-4858-9391-d12a1e32d8d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '216bcee1c0264184b196514311b8ce9d'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:22:02.698143 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-bccfb56f-4bbc-4858-9391-d12a1e32d8d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '216bcee1c0264184b196514311b8ce9d'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:22:02.737985 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1158/2335] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:22:02 2022] DELETE /identity/v3/users/216bcee1c0264184b196514311b8ce9d => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:22:03.351679 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c74573-130a-42b1-b3e5-1a461345e41d admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:22:03.353375 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-d7c74573-130a-42b1-b3e5-1a461345e41d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:22:03.372181 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c74573-130a-42b1-b3e5-1a461345e41d admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:22:03.464125 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07777ce4-de63-4cb5-9423-3ba14a994689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:22:03.464992 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07777ce4-de63-4cb5-9423-3ba14a994689 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:22:03.465365 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07777ce4-de63-4cb5-9423-3ba14a994689 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:22:03.465811 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07777ce4-de63-4cb5-9423-3ba14a994689 admin admin] PATH_INFO: `/v3/projects/b13388d4868f4e20864737abe9dbfbf8` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:22:03.467870 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07777ce4-de63-4cb5-9423-3ba14a994689 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b13388d4868f4e20864737abe9dbfbf8)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:22:03.470482 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07777ce4-de63-4cb5-9423-3ba14a994689 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:22:03.555235 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-07777ce4-de63-4cb5-9423-3ba14a994689 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b13388d4868f4e20864737abe9dbfbf8', 'request_id': 'req-07777ce4-de63-4cb5-9423-3ba14a994689'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:22:03.566874 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-07777ce4-de63-4cb5-9423-3ba14a994689 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b13388d4868f4e20864737abe9dbfbf8', 'request_id': 'req-07777ce4-de63-4cb5-9423-3ba14a994689'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:22:03.576655 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-07777ce4-de63-4cb5-9423-3ba14a994689 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b13388d4868f4e20864737abe9dbfbf8', 'request_id': 'req-07777ce4-de63-4cb5-9423-3ba14a994689'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:22:03.582843 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:22:03.597988 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-07777ce4-de63-4cb5-9423-3ba14a994689 admin admin] The token cache is being invalidate because project b13388d4868f4e20864737abe9dbfbf8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81772) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:22:03.598417 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-07777ce4-de63-4cb5-9423-3ba14a994689 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:22:03.599286 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-07777ce4-de63-4cb5-9423-3ba14a994689 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:22:03.600959 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1178/2336] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:22:03 2022] DELETE /identity/v3/projects/b13388d4868f4e20864737abe9dbfbf8 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 20:22:03.606782 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf2a3e9-02f2-4d84-a895-f26307436ca0 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:22:03.607644 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf2a3e9-02f2-4d84-a895-f26307436ca0 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:22:03.607644 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf2a3e9-02f2-4d84-a895-f26307436ca0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:22:03.634814 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-faf2a3e9-02f2-4d84-a895-f26307436ca0 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:22:03.637913 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-faf2a3e9-02f2-4d84-a895-f26307436ca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:22:03.726638 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:22:03.740096 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1159/2337] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:22:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:22:03.759290 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f45a98a-ed6a-421d-8166-b75bf7db1e78 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:22:03.759627 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f45a98a-ed6a-421d-8166-b75bf7db1e78 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:22:03.759843 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f45a98a-ed6a-421d-8166-b75bf7db1e78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:22:03.793456 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-0f45a98a-ed6a-421d-8166-b75bf7db1e78 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:22:03.795797 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-0f45a98a-ed6a-421d-8166-b75bf7db1e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:22:03.811565 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1179/2338] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:22:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:22:03.817852 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf2a3e9-02f2-4d84-a895-f26307436ca0 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:22:03.830962 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf2a3e9-02f2-4d84-a895-f26307436ca0 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:22:03.836793 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0493d7fe-f32b-41f2-9dc5-2d10c820ed05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:22:03.838213 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0493d7fe-f32b-41f2-9dc5-2d10c820ed05 admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:22:03.838805 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0493d7fe-f32b-41f2-9dc5-2d10c820ed05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:22:03.839932 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0493d7fe-f32b-41f2-9dc5-2d10c820ed05 admin admin] PATH_INFO: `/v3/domains` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:22:03.841344 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0493d7fe-f32b-41f2-9dc5-2d10c820ed05 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:22:03.842322 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0493d7fe-f32b-41f2-9dc5-2d10c820ed05 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:22:03.844906 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0493d7fe-f32b-41f2-9dc5-2d10c820ed05 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:22:03.864577 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1160/2339] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 6 20:22:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:22:03.869683 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f45a98a-ed6a-421d-8166-b75bf7db1e78 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:22:03.880926 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f45a98a-ed6a-421d-8166-b75bf7db1e78 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:22:03.887200 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1994f977-5409-46f7-9e60-de1274e514b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:22:03.887852 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1994f977-5409-46f7-9e60-de1274e514b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:22:03.888136 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1994f977-5409-46f7-9e60-de1274e514b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:22:03.888411 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1994f977-5409-46f7-9e60-de1274e514b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:22:03.889088 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1994f977-5409-46f7-9e60-de1274e514b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:22:03.891195 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1994f977-5409-46f7-9e60-de1274e514b8 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:22:03.936546 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1180/2340] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:22:03 2022] POST /identity/v3/projects => generated 361 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 20:22:03.941406 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0493d7fe-f32b-41f2-9dc5-2d10c820ed05 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:22:03.959062 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0493d7fe-f32b-41f2-9dc5-2d10c820ed05 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:22:04.080204 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0565bf21-9312-4b16-9598-190e32fc77a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:22:04.081106 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0565bf21-9312-4b16-9598-190e32fc77a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:22:04.081451 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0565bf21-9312-4b16-9598-190e32fc77a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:22:04.081827 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0565bf21-9312-4b16-9598-190e32fc77a9 admin admin] PATH_INFO: `/v3/users` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:22:04.082506 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0565bf21-9312-4b16-9598-190e32fc77a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:22:04.084890 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0565bf21-9312-4b16-9598-190e32fc77a9 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:22:04.117477 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1161/2341] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:22:03 2022] POST /identity/v3/users => generated 281 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 20:22:04.118935 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1994f977-5409-46f7-9e60-de1274e514b8 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:22:04.132026 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1994f977-5409-46f7-9e60-de1274e514b8 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:22:04.139777 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b2a904-c603-4d2b-968e-0a82d11e5496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:22:04.140441 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b2a904-c603-4d2b-968e-0a82d11e5496 admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:22:04.140710 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b2a904-c603-4d2b-968e-0a82d11e5496 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:22:04.140984 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b2a904-c603-4d2b-968e-0a82d11e5496 admin admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:22:04.141799 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b2a904-c603-4d2b-968e-0a82d11e5496 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:22:04.143478 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b2a904-c603-4d2b-968e-0a82d11e5496 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:22:04.161827 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1181/2342] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:22:04 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 20:22:04.166645 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0565bf21-9312-4b16-9598-190e32fc77a9 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:22:04.179754 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0565bf21-9312-4b16-9598-190e32fc77a9 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:22:04.186239 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16afe7c-82c7-4b75-8dfe-4066fa704755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:22:04.187056 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16afe7c-82c7-4b75-8dfe-4066fa704755 admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:22:04.187454 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16afe7c-82c7-4b75-8dfe-4066fa704755 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:22:04.187828 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16afe7c-82c7-4b75-8dfe-4066fa704755 admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:22:04.188620 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16afe7c-82c7-4b75-8dfe-4066fa704755 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:22:04.190798 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16afe7c-82c7-4b75-8dfe-4066fa704755 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:22:04.216490 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1162/2343] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:22:04 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:22:04.221528 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b2a904-c603-4d2b-968e-0a82d11e5496 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:22:04.237409 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b2a904-c603-4d2b-968e-0a82d11e5496 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:22:04.243858 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34047088-693b-46cc-b28e-88d0d4bb96a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:22:04.244791 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34047088-693b-46cc-b28e-88d0d4bb96a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:22:04.245076 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34047088-693b-46cc-b28e-88d0d4bb96a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:22:04.245362 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34047088-693b-46cc-b28e-88d0d4bb96a0 admin admin] PATH_INFO: `/v3/projects/21e38e9d674d46d5b63ed41484286a22/users/1e7176361ea5430591e1d89dbcb924b0/roles/4eea39e24b06431aa357053fe7907a3d` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:22:04.272351 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34047088-693b-46cc-b28e-88d0d4bb96a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21e38e9d674d46d5b63ed41484286a22, user_id=1e7176361ea5430591e1d89dbcb924b0, role_id=4eea39e24b06431aa357053fe7907a3d)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:22:04.277951 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34047088-693b-46cc-b28e-88d0d4bb96a0 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:22:04.297838 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1182/2344] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:22:04 2022] PUT /identity/v3/projects/21e38e9d674d46d5b63ed41484286a22/users/1e7176361ea5430591e1d89dbcb924b0/roles/4eea39e24b06431aa357053fe7907a3d => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:22:04.310327 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe123273-632b-4c55-be3a-d468639c996e None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:22:04.310815 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe123273-632b-4c55-be3a-d468639c996e None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:22:04.311274 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe123273-632b-4c55-be3a-d468639c996e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:22:04.352235 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-fe123273-632b-4c55-be3a-d468639c996e None None] MFA Rules not processed for user `1e7176361ea5430591e1d89dbcb924b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:22:04.353528 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-fe123273-632b-4c55-be3a-d468639c996e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:22:04.479305 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1163/2345] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:22:04 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:22:04.538268 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34047088-693b-46cc-b28e-88d0d4bb96a0 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:22:04.540889 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-34047088-693b-46cc-b28e-88d0d4bb96a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:22:04.648282 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34047088-693b-46cc-b28e-88d0d4bb96a0 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:22:04.655219 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1685bc1-6094-4ddd-978b-357a14bf2b34 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df1bca993bc142908cc7cf85eabd2b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:22:04.656296 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1685bc1-6094-4ddd-978b-357a14bf2b34 service cinder] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:22:04.656799 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1685bc1-6094-4ddd-978b-357a14bf2b34 service cinder] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:22:04.657309 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1685bc1-6094-4ddd-978b-357a14bf2b34 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:22:04.673452 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1685bc1-6094-4ddd-978b-357a14bf2b34 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:22:04.673452 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1685bc1-6094-4ddd-978b-357a14bf2b34 service cinder] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:22:04.683738 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1183/2346] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:22:04 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 20:22:04.704351 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe123273-632b-4c55-be3a-d468639c996e None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:22:04.788043 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:22:04.809963 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe123273-632b-4c55-be3a-d468639c996e None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:22:04.816419 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c98eaae-51c5-491c-8846-e81072b47d38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:22:04.817509 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c98eaae-51c5-491c-8846-e81072b47d38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:22:04.817910 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c98eaae-51c5-491c-8846-e81072b47d38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:22:04.818378 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c98eaae-51c5-491c-8846-e81072b47d38 admin admin] PATH_INFO: `/v3/users/1e7176361ea5430591e1d89dbcb924b0` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:22:04.820618 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c98eaae-51c5-491c-8846-e81072b47d38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e7176361ea5430591e1d89dbcb924b0)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:22:04.824109 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c98eaae-51c5-491c-8846-e81072b47d38 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:22:04.894576 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-7c98eaae-51c5-491c-8846-e81072b47d38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e7176361ea5430591e1d89dbcb924b0'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:22:04.903984 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-7c98eaae-51c5-491c-8846-e81072b47d38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e7176361ea5430591e1d89dbcb924b0'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:22:04.905080 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-7c98eaae-51c5-491c-8846-e81072b47d38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e7176361ea5430591e1d89dbcb924b0'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:22:04.905893 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-7c98eaae-51c5-491c-8846-e81072b47d38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e7176361ea5430591e1d89dbcb924b0'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:22:04.918893 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-7c98eaae-51c5-491c-8846-e81072b47d38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e7176361ea5430591e1d89dbcb924b0'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:22:04.964037 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1164/2347] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:22:04 2022] DELETE /identity/v3/users/1e7176361ea5430591e1d89dbcb924b0 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:22:04.969152 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df1f5f8-d09a-4bd9-95c8-9d03890e8ad6 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:22:04.969482 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df1f5f8-d09a-4bd9-95c8-9d03890e8ad6 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:22:04.969959 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df1f5f8-d09a-4bd9-95c8-9d03890e8ad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:22:05.003215 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-8df1f5f8-d09a-4bd9-95c8-9d03890e8ad6 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:22:05.005118 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-8df1f5f8-d09a-4bd9-95c8-9d03890e8ad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:22:05.097702 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1184/2348] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:22:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:22:05.111734 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c98eaae-51c5-491c-8846-e81072b47d38 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:22:05.114149 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-7c98eaae-51c5-491c-8846-e81072b47d38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:22:05.222849 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c98eaae-51c5-491c-8846-e81072b47d38 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:22:05.227353 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11740f9-d274-4a86-851f-ef095101054f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:22:05.228078 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11740f9-d274-4a86-851f-ef095101054f service neutron] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:22:05.228367 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11740f9-d274-4a86-851f-ef095101054f service neutron] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:22:05.228731 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11740f9-d274-4a86-851f-ef095101054f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:22:05.241473 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11740f9-d274-4a86-851f-ef095101054f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:22:05.243151 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11740f9-d274-4a86-851f-ef095101054f service neutron] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:22:05.256106 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1165/2349] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:22:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:22:05.718939 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df1f5f8-d09a-4bd9-95c8-9d03890e8ad6 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:22:05.722462 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-8df1f5f8-d09a-4bd9-95c8-9d03890e8ad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:22:05.752244 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df1f5f8-d09a-4bd9-95c8-9d03890e8ad6 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:22:05.762084 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47a6942-1111-4854-b70e-4b4fa0acf6d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:22:05.763242 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47a6942-1111-4854-b70e-4b4fa0acf6d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:22:05.764012 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47a6942-1111-4854-b70e-4b4fa0acf6d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:22:05.764568 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47a6942-1111-4854-b70e-4b4fa0acf6d7 admin admin] PATH_INFO: `/v3/projects/21e38e9d674d46d5b63ed41484286a22` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:22:05.766937 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47a6942-1111-4854-b70e-4b4fa0acf6d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=21e38e9d674d46d5b63ed41484286a22)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:22:05.772563 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47a6942-1111-4854-b70e-4b4fa0acf6d7 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:22:05.843031 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:22:05.871302 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-f47a6942-1111-4854-b70e-4b4fa0acf6d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21e38e9d674d46d5b63ed41484286a22', 'request_id': 'req-f47a6942-1111-4854-b70e-4b4fa0acf6d7'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:22:05.883101 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-f47a6942-1111-4854-b70e-4b4fa0acf6d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21e38e9d674d46d5b63ed41484286a22', 'request_id': 'req-f47a6942-1111-4854-b70e-4b4fa0acf6d7'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:22:05.893764 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-f47a6942-1111-4854-b70e-4b4fa0acf6d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21e38e9d674d46d5b63ed41484286a22', 'request_id': 'req-f47a6942-1111-4854-b70e-4b4fa0acf6d7'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:22:05.909988 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-f47a6942-1111-4854-b70e-4b4fa0acf6d7 admin admin] The token cache is being invalidate because project 21e38e9d674d46d5b63ed41484286a22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81772) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:22:05.910901 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-f47a6942-1111-4854-b70e-4b4fa0acf6d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:22:05.911803 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-f47a6942-1111-4854-b70e-4b4fa0acf6d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:22:05.913895 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1185/2350] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:22:05 2022] DELETE /identity/v3/projects/21e38e9d674d46d5b63ed41484286a22 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:22:05.920838 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5031bdb3-b098-4335-b836-8326900617d4 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:22:05.921436 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5031bdb3-b098-4335-b836-8326900617d4 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:22:05.921436 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5031bdb3-b098-4335-b836-8326900617d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:22:05.947049 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-5031bdb3-b098-4335-b836-8326900617d4 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:22:05.948789 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-5031bdb3-b098-4335-b836-8326900617d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:22:06.079688 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1166/2351] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:22:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:22:06.090720 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66016b7a-0dfc-4e12-b84f-b94bd62855d0 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:22:06.091129 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66016b7a-0dfc-4e12-b84f-b94bd62855d0 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:22:06.091499 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66016b7a-0dfc-4e12-b84f-b94bd62855d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:22:06.118536 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-66016b7a-0dfc-4e12-b84f-b94bd62855d0 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:22:06.121052 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-66016b7a-0dfc-4e12-b84f-b94bd62855d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:22:06.134670 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1186/2352] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:22:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:22:06.139599 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5031bdb3-b098-4335-b836-8326900617d4 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:22:06.151389 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5031bdb3-b098-4335-b836-8326900617d4 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:22:06.158237 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbf46c9-e499-4b1a-94e7-3384670858f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:22:06.158966 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbf46c9-e499-4b1a-94e7-3384670858f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:22:06.159387 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbf46c9-e499-4b1a-94e7-3384670858f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:22:06.159756 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbf46c9-e499-4b1a-94e7-3384670858f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:22:06.160597 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbf46c9-e499-4b1a-94e7-3384670858f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:22:06.160953 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbf46c9-e499-4b1a-94e7-3384670858f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:22:06.162590 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbf46c9-e499-4b1a-94e7-3384670858f2 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:22:06.181818 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1167/2353] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 6 20:22:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:22:06.187473 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66016b7a-0dfc-4e12-b84f-b94bd62855d0 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:22:06.199995 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66016b7a-0dfc-4e12-b84f-b94bd62855d0 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:22:06.205492 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2ec3f1-9d8b-4b1e-ac92-480ea5bc20c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:22:06.206158 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2ec3f1-9d8b-4b1e-ac92-480ea5bc20c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:22:06.206656 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2ec3f1-9d8b-4b1e-ac92-480ea5bc20c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:22:06.206834 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2ec3f1-9d8b-4b1e-ac92-480ea5bc20c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:22:06.207352 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2ec3f1-9d8b-4b1e-ac92-480ea5bc20c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:22:06.209463 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2ec3f1-9d8b-4b1e-ac92-480ea5bc20c4 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:22:06.249881 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1187/2354] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:22:06 2022] POST /identity/v3/projects => generated 377 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:22:06.254496 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbf46c9-e499-4b1a-94e7-3384670858f2 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:22:06.260497 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:22:06.267328 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbf46c9-e499-4b1a-94e7-3384670858f2 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:22:06.347325 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c98694-6b79-4f1a-9879-898378ea7098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:22:06.348129 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c98694-6b79-4f1a-9879-898378ea7098 admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:22:06.348470 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c98694-6b79-4f1a-9879-898378ea7098 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:22:06.348756 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c98694-6b79-4f1a-9879-898378ea7098 admin admin] PATH_INFO: `/v3/users` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:22:06.349464 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c98694-6b79-4f1a-9879-898378ea7098 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:22:06.351774 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c98694-6b79-4f1a-9879-898378ea7098 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:22:06.390959 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1168/2355] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:22:06 2022] POST /identity/v3/users => generated 289 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:22:06.396338 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2ec3f1-9d8b-4b1e-ac92-480ea5bc20c4 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:22:06.419879 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2ec3f1-9d8b-4b1e-ac92-480ea5bc20c4 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:22:06.428942 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a0fef8-02ed-4aba-9635-a9df3e8054a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:22:06.430047 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a0fef8-02ed-4aba-9635-a9df3e8054a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:22:06.430530 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a0fef8-02ed-4aba-9635-a9df3e8054a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:22:06.431022 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a0fef8-02ed-4aba-9635-a9df3e8054a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:22:06.432340 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a0fef8-02ed-4aba-9635-a9df3e8054a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:22:06.435642 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a0fef8-02ed-4aba-9635-a9df3e8054a5 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:22:06.462149 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1188/2356] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:22:06 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:22:06.464317 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c98694-6b79-4f1a-9879-898378ea7098 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:22:06.484835 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c98694-6b79-4f1a-9879-898378ea7098 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:22:06.492923 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33e8497-b04f-4676-99da-940bac0637c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:22:06.494108 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33e8497-b04f-4676-99da-940bac0637c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:22:06.494500 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33e8497-b04f-4676-99da-940bac0637c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:22:06.494895 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33e8497-b04f-4676-99da-940bac0637c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:22:06.495786 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33e8497-b04f-4676-99da-940bac0637c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:22:06.497917 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33e8497-b04f-4676-99da-940bac0637c1 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:22:06.511960 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1169/2357] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:22:06 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:22:06.516928 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a0fef8-02ed-4aba-9635-a9df3e8054a5 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:22:06.529568 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a0fef8-02ed-4aba-9635-a9df3e8054a5 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:22:06.535408 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c88f3d-dcc2-4df0-ab91-f52b3ce3d248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:22:06.535846 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c88f3d-dcc2-4df0-ab91-f52b3ce3d248 admin admin] REQUEST_METHOD: `PUT` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:22:06.536129 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c88f3d-dcc2-4df0-ab91-f52b3ce3d248 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:22:06.536408 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c88f3d-dcc2-4df0-ab91-f52b3ce3d248 admin admin] PATH_INFO: `/v3/projects/37cd06eca7aa4959aad2963b9ed91124/users/c25f1a0ab3f7462bb5f4a3764d12d412/roles/4eea39e24b06431aa357053fe7907a3d` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:22:06.554898 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c88f3d-dcc2-4df0-ab91-f52b3ce3d248 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37cd06eca7aa4959aad2963b9ed91124, user_id=c25f1a0ab3f7462bb5f4a3764d12d412, role_id=4eea39e24b06431aa357053fe7907a3d)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:22:06.557882 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c88f3d-dcc2-4df0-ab91-f52b3ce3d248 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:22:06.578277 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1189/2358] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:22:06 2022] PUT /identity/v3/projects/37cd06eca7aa4959aad2963b9ed91124/users/c25f1a0ab3f7462bb5f4a3764d12d412/roles/4eea39e24b06431aa357053fe7907a3d => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:22:06.598524 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a17fa4-15b9-4512-8fa5-d7c5772bb6ed None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:22:06.599047 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a17fa4-15b9-4512-8fa5-d7c5772bb6ed None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:22:06.599539 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a17fa4-15b9-4512-8fa5-d7c5772bb6ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:22:06.649443 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-17a17fa4-15b9-4512-8fa5-d7c5772bb6ed None None] MFA Rules not processed for user `c25f1a0ab3f7462bb5f4a3764d12d412`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:22:06.651054 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-17a17fa4-15b9-4512-8fa5-d7c5772bb6ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:22:06.760668 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1170/2359] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:22:06 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:22:06.775431 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c88f3d-dcc2-4df0-ab91-f52b3ce3d248 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:22:06.777913 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-81c88f3d-dcc2-4df0-ab91-f52b3ce3d248 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:22:06.862755 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c88f3d-dcc2-4df0-ab91-f52b3ce3d248 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:22:06.867595 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7779836-0686-4449-afdd-dd8a0bc3a10b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df1bca993bc142908cc7cf85eabd2b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:22:06.868565 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7779836-0686-4449-afdd-dd8a0bc3a10b service cinder] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:22:06.868859 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7779836-0686-4449-afdd-dd8a0bc3a10b service cinder] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:22:06.869260 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7779836-0686-4449-afdd-dd8a0bc3a10b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:22:06.882891 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7779836-0686-4449-afdd-dd8a0bc3a10b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:22:06.885465 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7779836-0686-4449-afdd-dd8a0bc3a10b service cinder] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:22:06.896647 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1190/2360] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:22:06 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:22:07.647233 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea122c28-b0b0-4e7d-b723-1a8736c03a40 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:22:07.648082 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea122c28-b0b0-4e7d-b723-1a8736c03a40 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:22:07.648252 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea122c28-b0b0-4e7d-b723-1a8736c03a40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:22:07.680258 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-ea122c28-b0b0-4e7d-b723-1a8736c03a40 None None] MFA Rules not processed for user `ad400ce73b644938aef19e2da731c4dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:22:07.681934 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-ea122c28-b0b0-4e7d-b723-1a8736c03a40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:22:07.777258 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1171/2361] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:22:07 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:22:16.892261 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=67,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:22:17.769012 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=90,keystone:INSERT=3 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:22:27.200040 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7779836-0686-4449-afdd-dd8a0bc3a10b service cinder] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:22:27.291252 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7779836-0686-4449-afdd-dd8a0bc3a10b service cinder] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:22:27.299299 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c2c063-1fcc-4036-9576-b9484e936aa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:22:27.300217 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c2c063-1fcc-4036-9576-b9484e936aa1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:22:27.300624 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c2c063-1fcc-4036-9576-b9484e936aa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:22:27.301041 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c2c063-1fcc-4036-9576-b9484e936aa1 admin admin] PATH_INFO: `/v3/users/c25f1a0ab3f7462bb5f4a3764d12d412` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:22:27.303141 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c2c063-1fcc-4036-9576-b9484e936aa1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c25f1a0ab3f7462bb5f4a3764d12d412)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:22:27.306903 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c2c063-1fcc-4036-9576-b9484e936aa1 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:22:27.386689 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-c9c2c063-1fcc-4036-9576-b9484e936aa1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c25f1a0ab3f7462bb5f4a3764d12d412'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:22:27.387510 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-c9c2c063-1fcc-4036-9576-b9484e936aa1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c25f1a0ab3f7462bb5f4a3764d12d412'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:22:27.406228 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-c9c2c063-1fcc-4036-9576-b9484e936aa1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c25f1a0ab3f7462bb5f4a3764d12d412'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:22:27.407594 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-c9c2c063-1fcc-4036-9576-b9484e936aa1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c25f1a0ab3f7462bb5f4a3764d12d412'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:22:27.418888 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-c9c2c063-1fcc-4036-9576-b9484e936aa1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c25f1a0ab3f7462bb5f4a3764d12d412'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:22:27.444882 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1191/2362] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:22:27 2022] DELETE /identity/v3/users/c25f1a0ab3f7462bb5f4a3764d12d412 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:22:27.451998 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04918a01-136a-43f1-91b3-298209edc471 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:22:27.454434 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04918a01-136a-43f1-91b3-298209edc471 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:22:27.455662 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04918a01-136a-43f1-91b3-298209edc471 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:22:27.491819 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-04918a01-136a-43f1-91b3-298209edc471 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:22:27.495348 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-04918a01-136a-43f1-91b3-298209edc471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:22:27.606172 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1172/2363] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:22:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:22:27.618556 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c2c063-1fcc-4036-9576-b9484e936aa1 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:22:27.621148 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-c9c2c063-1fcc-4036-9576-b9484e936aa1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:22:27.738009 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c2c063-1fcc-4036-9576-b9484e936aa1 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:22:27.743538 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434c1816-cf5c-4862-a294-6c765b17b7e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:22:27.744986 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434c1816-cf5c-4862-a294-6c765b17b7e3 service neutron] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:22:27.745413 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434c1816-cf5c-4862-a294-6c765b17b7e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:22:27.745869 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434c1816-cf5c-4862-a294-6c765b17b7e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:22:27.764016 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434c1816-cf5c-4862-a294-6c765b17b7e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:22:27.767406 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434c1816-cf5c-4862-a294-6c765b17b7e3 service neutron] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:22:27.789924 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1192/2364] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:22:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:22:28.731011 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04918a01-136a-43f1-91b3-298209edc471 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:22:28.732760 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-04918a01-136a-43f1-91b3-298209edc471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:22:28.752009 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04918a01-136a-43f1-91b3-298209edc471 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:22:28.758581 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36774943-8842-45ed-a8a1-e7b54345762f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:22:28.759237 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36774943-8842-45ed-a8a1-e7b54345762f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:22:28.760170 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36774943-8842-45ed-a8a1-e7b54345762f admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:22:28.760450 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36774943-8842-45ed-a8a1-e7b54345762f admin admin] PATH_INFO: `/v3/projects/37cd06eca7aa4959aad2963b9ed91124` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:22:28.762448 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36774943-8842-45ed-a8a1-e7b54345762f admin admin] RBAC: Authorizing `identity:delete_project(project_id=37cd06eca7aa4959aad2963b9ed91124)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:22:28.765319 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36774943-8842-45ed-a8a1-e7b54345762f admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:22:28.832634 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-36774943-8842-45ed-a8a1-e7b54345762f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37cd06eca7aa4959aad2963b9ed91124', 'request_id': 'req-36774943-8842-45ed-a8a1-e7b54345762f'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:22:28.840212 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-36774943-8842-45ed-a8a1-e7b54345762f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37cd06eca7aa4959aad2963b9ed91124', 'request_id': 'req-36774943-8842-45ed-a8a1-e7b54345762f'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:22:28.852955 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-36774943-8842-45ed-a8a1-e7b54345762f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37cd06eca7aa4959aad2963b9ed91124', 'request_id': 'req-36774943-8842-45ed-a8a1-e7b54345762f'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:22:28.871781 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-36774943-8842-45ed-a8a1-e7b54345762f admin admin] The token cache is being invalidate because project 37cd06eca7aa4959aad2963b9ed91124 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81771) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:22:28.872128 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-36774943-8842-45ed-a8a1-e7b54345762f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:22:28.872940 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-36774943-8842-45ed-a8a1-e7b54345762f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:22:28.875409 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1173/2365] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:22:28 2022] DELETE /identity/v3/projects/37cd06eca7aa4959aad2963b9ed91124 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:22:28.880541 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836b0949-a963-4abc-83d5-6d87ea2f9d66 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:22:28.881154 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836b0949-a963-4abc-83d5-6d87ea2f9d66 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:22:28.881154 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836b0949-a963-4abc-83d5-6d87ea2f9d66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:22:28.907485 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-836b0949-a963-4abc-83d5-6d87ea2f9d66 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:22:28.909720 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-836b0949-a963-4abc-83d5-6d87ea2f9d66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:22:28.940809 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:22:29.020545 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1193/2366] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:22:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:22:29.035672 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e82fbf-8887-4ab0-8167-fefee4d1b008 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:22:29.036061 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e82fbf-8887-4ab0-8167-fefee4d1b008 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:22:29.036399 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e82fbf-8887-4ab0-8167-fefee4d1b008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:22:29.063725 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-10e82fbf-8887-4ab0-8167-fefee4d1b008 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:22:29.066521 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-10e82fbf-8887-4ab0-8167-fefee4d1b008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:22:29.079779 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1174/2367] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:22:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:22:29.088723 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836b0949-a963-4abc-83d5-6d87ea2f9d66 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:22:29.104226 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836b0949-a963-4abc-83d5-6d87ea2f9d66 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:22:29.110175 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5d5d6b-b92a-4b48-9319-2fa30f683dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:22:29.110992 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5d5d6b-b92a-4b48-9319-2fa30f683dc1 admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:22:29.111383 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5d5d6b-b92a-4b48-9319-2fa30f683dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:22:29.111805 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5d5d6b-b92a-4b48-9319-2fa30f683dc1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:22:29.112643 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5d5d6b-b92a-4b48-9319-2fa30f683dc1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:22:29.113264 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5d5d6b-b92a-4b48-9319-2fa30f683dc1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:22:29.115888 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5d5d6b-b92a-4b48-9319-2fa30f683dc1 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:22:29.132609 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1194/2368] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 6 20:22:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:22:29.138334 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e82fbf-8887-4ab0-8167-fefee4d1b008 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:22:29.148451 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e82fbf-8887-4ab0-8167-fefee4d1b008 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:22:29.154936 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f377d9e-cff5-4b11-81f1-f9beff80995a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:22:29.155678 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f377d9e-cff5-4b11-81f1-f9beff80995a admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:22:29.156015 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f377d9e-cff5-4b11-81f1-f9beff80995a admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:22:29.156375 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f377d9e-cff5-4b11-81f1-f9beff80995a admin admin] PATH_INFO: `/v3/projects` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:22:29.156966 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f377d9e-cff5-4b11-81f1-f9beff80995a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:22:29.159227 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f377d9e-cff5-4b11-81f1-f9beff80995a admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:22:29.194939 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1175/2369] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:22:29 2022] POST /identity/v3/projects => generated 377 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:22:29.199382 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5d5d6b-b92a-4b48-9319-2fa30f683dc1 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:22:29.211590 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5d5d6b-b92a-4b48-9319-2fa30f683dc1 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:22:29.294577 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93f1ed1-549b-4e88-a312-b4d1ee6f37e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:22:29.295218 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93f1ed1-549b-4e88-a312-b4d1ee6f37e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:22:29.295558 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93f1ed1-549b-4e88-a312-b4d1ee6f37e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:22:29.295879 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93f1ed1-549b-4e88-a312-b4d1ee6f37e5 admin admin] PATH_INFO: `/v3/users` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:22:29.296694 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93f1ed1-549b-4e88-a312-b4d1ee6f37e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:22:29.298782 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93f1ed1-549b-4e88-a312-b4d1ee6f37e5 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:22:29.329226 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1195/2370] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:22:29 2022] POST /identity/v3/users => generated 289 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 20:22:29.334487 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f377d9e-cff5-4b11-81f1-f9beff80995a admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:22:29.347149 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f377d9e-cff5-4b11-81f1-f9beff80995a admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:22:29.352811 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf066ba-c48a-41eb-8caa-9e0bac5ecb5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:22:29.353581 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf066ba-c48a-41eb-8caa-9e0bac5ecb5e admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:22:29.353885 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf066ba-c48a-41eb-8caa-9e0bac5ecb5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:22:29.354159 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf066ba-c48a-41eb-8caa-9e0bac5ecb5e admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:22:29.354884 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf066ba-c48a-41eb-8caa-9e0bac5ecb5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:22:29.356472 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf066ba-c48a-41eb-8caa-9e0bac5ecb5e admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:22:29.368266 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1176/2371] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:22:29 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:22:29.373506 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93f1ed1-549b-4e88-a312-b4d1ee6f37e5 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:22:29.389669 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93f1ed1-549b-4e88-a312-b4d1ee6f37e5 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:22:29.400944 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9274ee5b-5236-4a91-b030-e4c75f88443e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:22:29.402031 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9274ee5b-5236-4a91-b030-e4c75f88443e admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:22:29.402490 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9274ee5b-5236-4a91-b030-e4c75f88443e admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:22:29.403780 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9274ee5b-5236-4a91-b030-e4c75f88443e admin admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:22:29.405685 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9274ee5b-5236-4a91-b030-e4c75f88443e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:22:29.408171 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9274ee5b-5236-4a91-b030-e4c75f88443e admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:22:29.421907 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1196/2372] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:22:29 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 20:22:29.427374 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf066ba-c48a-41eb-8caa-9e0bac5ecb5e admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:22:29.446825 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf066ba-c48a-41eb-8caa-9e0bac5ecb5e admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:22:29.455510 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b41121-5af8-4249-945e-d53ccf246015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:22:29.456704 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b41121-5af8-4249-945e-d53ccf246015 admin admin] REQUEST_METHOD: `PUT` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:22:29.457253 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b41121-5af8-4249-945e-d53ccf246015 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:22:29.457845 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b41121-5af8-4249-945e-d53ccf246015 admin admin] PATH_INFO: `/v3/projects/b1b4a49134f34659af42d3bc351e6fd8/users/1c40d925187c43e996a4bd03dfd4fde6/roles/4eea39e24b06431aa357053fe7907a3d` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:22:29.479555 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b41121-5af8-4249-945e-d53ccf246015 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1b4a49134f34659af42d3bc351e6fd8, user_id=1c40d925187c43e996a4bd03dfd4fde6, role_id=4eea39e24b06431aa357053fe7907a3d)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:22:29.483103 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b41121-5af8-4249-945e-d53ccf246015 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:22:29.502836 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1177/2373] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:22:29 2022] PUT /identity/v3/projects/b1b4a49134f34659af42d3bc351e6fd8/users/1c40d925187c43e996a4bd03dfd4fde6/roles/4eea39e24b06431aa357053fe7907a3d => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:22:29.577595 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450f8b3d-f32c-4f62-aa61-3ff5e7a75beb None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:22:29.578453 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450f8b3d-f32c-4f62-aa61-3ff5e7a75beb None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:22:29.579046 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450f8b3d-f32c-4f62-aa61-3ff5e7a75beb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:22:29.618600 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-450f8b3d-f32c-4f62-aa61-3ff5e7a75beb None None] MFA Rules not processed for user `1c40d925187c43e996a4bd03dfd4fde6`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:22:29.620289 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-450f8b3d-f32c-4f62-aa61-3ff5e7a75beb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:22:29.767299 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1197/2374] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:22:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:22:29.768592 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b41121-5af8-4249-945e-d53ccf246015 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:22:29.804106 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:22:29.920201 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b41121-5af8-4249-945e-d53ccf246015 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:22:29.938017 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939adf1b-8684-4426-99e2-8432ac163e5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:22:29.939623 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939adf1b-8684-4426-99e2-8432ac163e5b admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:22:29.940439 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939adf1b-8684-4426-99e2-8432ac163e5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:22:29.941043 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939adf1b-8684-4426-99e2-8432ac163e5b admin admin] PATH_INFO: `/v3/projects` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:22:29.942855 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939adf1b-8684-4426-99e2-8432ac163e5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:22:29.947943 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939adf1b-8684-4426-99e2-8432ac163e5b admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:22:30.036427 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1178/2375] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:22:29 2022] POST /identity/v3/projects => generated 377 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:22:30.037230 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450f8b3d-f32c-4f62-aa61-3ff5e7a75beb None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:22:30.052197 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450f8b3d-f32c-4f62-aa61-3ff5e7a75beb None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:22:30.075045 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:22:30.149331 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e0f241-8f72-4143-82d4-23328a758fa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:22:30.151331 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e0f241-8f72-4143-82d4-23328a758fa4 admin admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:22:30.152033 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e0f241-8f72-4143-82d4-23328a758fa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:22:30.152872 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e0f241-8f72-4143-82d4-23328a758fa4 admin admin] PATH_INFO: `/v3/users` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:22:30.154564 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e0f241-8f72-4143-82d4-23328a758fa4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:22:30.158840 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e0f241-8f72-4143-82d4-23328a758fa4 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:22:30.200059 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1198/2376] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:22:30 2022] POST /identity/v3/users => generated 289 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 20:22:30.204404 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939adf1b-8684-4426-99e2-8432ac163e5b admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:22:30.218381 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939adf1b-8684-4426-99e2-8432ac163e5b admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:22:30.223887 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fc224a-aa90-4cad-b836-9dc7e5d1a863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:22:30.228480 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fc224a-aa90-4cad-b836-9dc7e5d1a863 admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:22:30.228480 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fc224a-aa90-4cad-b836-9dc7e5d1a863 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:22:30.228480 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fc224a-aa90-4cad-b836-9dc7e5d1a863 admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:22:30.228480 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fc224a-aa90-4cad-b836-9dc7e5d1a863 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:22:30.238421 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fc224a-aa90-4cad-b836-9dc7e5d1a863 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:22:30.258968 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1179/2377] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:22:30 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:22:30.261341 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e0f241-8f72-4143-82d4-23328a758fa4 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:22:30.282498 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e0f241-8f72-4143-82d4-23328a758fa4 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:22:30.291007 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c65b318-66d6-4351-a7f4-67fc8854d46d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:22:30.291248 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c65b318-66d6-4351-a7f4-67fc8854d46d admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:22:30.292074 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c65b318-66d6-4351-a7f4-67fc8854d46d admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:22:30.292256 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c65b318-66d6-4351-a7f4-67fc8854d46d admin admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:22:30.293658 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c65b318-66d6-4351-a7f4-67fc8854d46d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:22:30.296639 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c65b318-66d6-4351-a7f4-67fc8854d46d admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:22:30.318253 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1199/2378] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:22:30 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:22:30.323258 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fc224a-aa90-4cad-b836-9dc7e5d1a863 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:22:30.338153 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fc224a-aa90-4cad-b836-9dc7e5d1a863 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:22:30.344428 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fbc468-ac02-4ba1-af8d-c4ac510afe90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:22:30.345306 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fbc468-ac02-4ba1-af8d-c4ac510afe90 admin admin] REQUEST_METHOD: `PUT` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:22:30.345594 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fbc468-ac02-4ba1-af8d-c4ac510afe90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:22:30.346060 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fbc468-ac02-4ba1-af8d-c4ac510afe90 admin admin] PATH_INFO: `/v3/projects/faa571e2bf5a457eabb0ea94032a5850/users/8cd6045b5f5d4e43b752e703a3fc6cea/roles/4eea39e24b06431aa357053fe7907a3d` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:22:30.367687 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fbc468-ac02-4ba1-af8d-c4ac510afe90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=faa571e2bf5a457eabb0ea94032a5850, user_id=8cd6045b5f5d4e43b752e703a3fc6cea, role_id=4eea39e24b06431aa357053fe7907a3d)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:22:30.370592 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fbc468-ac02-4ba1-af8d-c4ac510afe90 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:22:30.418935 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1180/2379] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:22:30 2022] PUT /identity/v3/projects/faa571e2bf5a457eabb0ea94032a5850/users/8cd6045b5f5d4e43b752e703a3fc6cea/roles/4eea39e24b06431aa357053fe7907a3d => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 20:22:30.438897 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db8c0d5-ca13-49b1-b9f8-80c9a44dde93 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:22:30.439817 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db8c0d5-ca13-49b1-b9f8-80c9a44dde93 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:22:30.440325 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db8c0d5-ca13-49b1-b9f8-80c9a44dde93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:22:30.496909 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-7db8c0d5-ca13-49b1-b9f8-80c9a44dde93 None None] MFA Rules not processed for user `8cd6045b5f5d4e43b752e703a3fc6cea`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:22:30.499177 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-7db8c0d5-ca13-49b1-b9f8-80c9a44dde93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:22:30.618133 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1200/2380] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:22:30 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:22:30.620560 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fbc468-ac02-4ba1-af8d-c4ac510afe90 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:22:30.730842 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fbc468-ac02-4ba1-af8d-c4ac510afe90 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:22:30.739326 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84781c79-d7f1-4511-91be-aae67e0719fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:22:30.740765 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84781c79-d7f1-4511-91be-aae67e0719fb admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:22:30.741241 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84781c79-d7f1-4511-91be-aae67e0719fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:22:30.741742 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84781c79-d7f1-4511-91be-aae67e0719fb admin admin] PATH_INFO: `/v3/projects` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:22:30.742586 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84781c79-d7f1-4511-91be-aae67e0719fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:22:30.746695 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84781c79-d7f1-4511-91be-aae67e0719fb admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:22:30.784248 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1181/2381] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:22:30 2022] POST /identity/v3/projects => generated 377 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:22:30.790921 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db8c0d5-ca13-49b1-b9f8-80c9a44dde93 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:22:30.808617 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db8c0d5-ca13-49b1-b9f8-80c9a44dde93 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:22:30.920468 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06099ba-1e1e-4461-a039-32e68c2acde2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:22:30.921540 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06099ba-1e1e-4461-a039-32e68c2acde2 admin admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:22:30.922087 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06099ba-1e1e-4461-a039-32e68c2acde2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:22:30.922588 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06099ba-1e1e-4461-a039-32e68c2acde2 admin admin] PATH_INFO: `/v3/users` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:22:30.923905 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06099ba-1e1e-4461-a039-32e68c2acde2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:22:30.927188 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06099ba-1e1e-4461-a039-32e68c2acde2 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:22:30.960113 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1201/2382] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:22:30 2022] POST /identity/v3/users => generated 295 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:22:30.964292 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84781c79-d7f1-4511-91be-aae67e0719fb admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:22:30.976559 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84781c79-d7f1-4511-91be-aae67e0719fb admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:22:30.983208 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0706bf-7593-4cc5-b5aa-c1df11750fea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:22:30.983208 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0706bf-7593-4cc5-b5aa-c1df11750fea admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:22:30.983208 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0706bf-7593-4cc5-b5aa-c1df11750fea admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:22:30.983872 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0706bf-7593-4cc5-b5aa-c1df11750fea admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:22:30.984042 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0706bf-7593-4cc5-b5aa-c1df11750fea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:22:30.986085 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0706bf-7593-4cc5-b5aa-c1df11750fea admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:22:31.004120 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1182/2383] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:22:30 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:22:31.008890 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06099ba-1e1e-4461-a039-32e68c2acde2 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:22:31.025989 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06099ba-1e1e-4461-a039-32e68c2acde2 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:22:31.033295 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b7d9c2-3f3a-4309-adc3-3685d330645d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:22:31.034226 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b7d9c2-3f3a-4309-adc3-3685d330645d admin admin] REQUEST_METHOD: `PUT` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:22:31.034546 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b7d9c2-3f3a-4309-adc3-3685d330645d admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:22:31.034889 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b7d9c2-3f3a-4309-adc3-3685d330645d admin admin] PATH_INFO: `/v3/projects/d794ca28ed7a4627b87f7c5400ec998c/users/e0a52b8da9624be8ab62dea6fee92aff/roles/e0ab41d8ae22474688b5361bcc75e0f0` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:22:31.050814 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:22:31.055154 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b7d9c2-3f3a-4309-adc3-3685d330645d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d794ca28ed7a4627b87f7c5400ec998c, user_id=e0a52b8da9624be8ab62dea6fee92aff, role_id=e0ab41d8ae22474688b5361bcc75e0f0)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:22:31.058501 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b7d9c2-3f3a-4309-adc3-3685d330645d admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:22:31.080076 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1202/2384] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:22:31 2022] PUT /identity/v3/projects/d794ca28ed7a4627b87f7c5400ec998c/users/e0a52b8da9624be8ab62dea6fee92aff/roles/e0ab41d8ae22474688b5361bcc75e0f0 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:22:31.086149 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0706bf-7593-4cc5-b5aa-c1df11750fea admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:22:31.104405 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0706bf-7593-4cc5-b5aa-c1df11750fea admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:22:31.177227 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:22:31.207479 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4215f9c0-5760-4ac6-a99a-fb58ac5acdaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:22:31.208748 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4215f9c0-5760-4ac6-a99a-fb58ac5acdaf admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:22:31.208748 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4215f9c0-5760-4ac6-a99a-fb58ac5acdaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:22:31.209106 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4215f9c0-5760-4ac6-a99a-fb58ac5acdaf admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:22:31.210294 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4215f9c0-5760-4ac6-a99a-fb58ac5acdaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:22:31.213650 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4215f9c0-5760-4ac6-a99a-fb58ac5acdaf admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:22:31.241571 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1183/2385] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:22:31 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:22:31.243637 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b7d9c2-3f3a-4309-adc3-3685d330645d admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:22:31.261254 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b7d9c2-3f3a-4309-adc3-3685d330645d admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:22:31.267950 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036ebddd-60ec-4122-abe7-813136a6fed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:22:31.268789 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036ebddd-60ec-4122-abe7-813136a6fed9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:22:31.269252 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036ebddd-60ec-4122-abe7-813136a6fed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:22:31.269753 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036ebddd-60ec-4122-abe7-813136a6fed9 admin admin] PATH_INFO: `/v3/projects/d794ca28ed7a4627b87f7c5400ec998c/users/e0a52b8da9624be8ab62dea6fee92aff/roles/e0ab41d8ae22474688b5361bcc75e0f0` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:22:31.273057 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036ebddd-60ec-4122-abe7-813136a6fed9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d794ca28ed7a4627b87f7c5400ec998c, user_id=e0a52b8da9624be8ab62dea6fee92aff, role_id=e0ab41d8ae22474688b5361bcc75e0f0)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:22:31.277101 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036ebddd-60ec-4122-abe7-813136a6fed9 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:22:31.302076 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1203/2386] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:22:31 2022] PUT /identity/v3/projects/d794ca28ed7a4627b87f7c5400ec998c/users/e0a52b8da9624be8ab62dea6fee92aff/roles/e0ab41d8ae22474688b5361bcc75e0f0 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:22:31.315597 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb66228d-62c8-4a51-a880-9b230e13bc5e None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:22:31.316034 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb66228d-62c8-4a51-a880-9b230e13bc5e None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:22:31.316446 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb66228d-62c8-4a51-a880-9b230e13bc5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:22:31.353181 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-fb66228d-62c8-4a51-a880-9b230e13bc5e None None] MFA Rules not processed for user `e0a52b8da9624be8ab62dea6fee92aff`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:22:31.354815 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-fb66228d-62c8-4a51-a880-9b230e13bc5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:22:31.490106 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1184/2387] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:22:31 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:22:31.512549 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036ebddd-60ec-4122-abe7-813136a6fed9 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:22:31.515313 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-036ebddd-60ec-4122-abe7-813136a6fed9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:22:31.654366 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036ebddd-60ec-4122-abe7-813136a6fed9 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:22:31.661263 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720e1b9b-104f-46ab-bcfa-e15c95c05a83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df1bca993bc142908cc7cf85eabd2b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:22:31.662815 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720e1b9b-104f-46ab-bcfa-e15c95c05a83 service cinder] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:22:31.663062 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720e1b9b-104f-46ab-bcfa-e15c95c05a83 service cinder] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:22:31.663433 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720e1b9b-104f-46ab-bcfa-e15c95c05a83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:22:31.751539 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720e1b9b-104f-46ab-bcfa-e15c95c05a83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:22:31.753088 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720e1b9b-104f-46ab-bcfa-e15c95c05a83 service cinder] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:22:31.767345 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1204/2388] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:22:31 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:22:33.557355 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb66228d-62c8-4a51-a880-9b230e13bc5e None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:22:33.559603 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-fb66228d-62c8-4a51-a880-9b230e13bc5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:22:33.574866 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb66228d-62c8-4a51-a880-9b230e13bc5e None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:22:33.579322 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33864490-6e72-4009-9b17-d15b33ae6c37 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df1bca993bc142908cc7cf85eabd2b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:22:33.579952 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33864490-6e72-4009-9b17-d15b33ae6c37 service cinder] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:22:33.580348 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33864490-6e72-4009-9b17-d15b33ae6c37 service cinder] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:22:33.580648 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33864490-6e72-4009-9b17-d15b33ae6c37 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:22:33.682699 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33864490-6e72-4009-9b17-d15b33ae6c37 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:22:33.684257 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33864490-6e72-4009-9b17-d15b33ae6c37 service cinder] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:22:33.694695 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1185/2389] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:22:33 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 20:22:34.537122 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720e1b9b-104f-46ab-bcfa-e15c95c05a83 service cinder] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:22:34.547944 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720e1b9b-104f-46ab-bcfa-e15c95c05a83 service cinder] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:22:34.552269 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66be3b60-e2eb-4b61-99e5-9ada5ab3db2a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df1bca993bc142908cc7cf85eabd2b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:22:34.553014 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66be3b60-e2eb-4b61-99e5-9ada5ab3db2a service cinder] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:22:34.553322 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66be3b60-e2eb-4b61-99e5-9ada5ab3db2a service cinder] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:22:34.553663 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66be3b60-e2eb-4b61-99e5-9ada5ab3db2a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:22:34.564842 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66be3b60-e2eb-4b61-99e5-9ada5ab3db2a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:22:34.566442 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66be3b60-e2eb-4b61-99e5-9ada5ab3db2a service cinder] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:22:34.578907 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1205/2390] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:22:34 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:22:39.568683 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33864490-6e72-4009-9b17-d15b33ae6c37 service cinder] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:22:39.671711 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33864490-6e72-4009-9b17-d15b33ae6c37 service cinder] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:22:39.677315 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f8018c-d2fe-47a8-ae4c-b5797bf36489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:22:39.678208 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f8018c-d2fe-47a8-ae4c-b5797bf36489 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:22:39.678452 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f8018c-d2fe-47a8-ae4c-b5797bf36489 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:22:39.678732 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f8018c-d2fe-47a8-ae4c-b5797bf36489 admin admin] PATH_INFO: `/v3/users/1c40d925187c43e996a4bd03dfd4fde6` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:22:39.679914 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f8018c-d2fe-47a8-ae4c-b5797bf36489 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c40d925187c43e996a4bd03dfd4fde6)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:22:39.682640 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f8018c-d2fe-47a8-ae4c-b5797bf36489 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:22:39.688808 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:22:39.768257 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-d5f8018c-d2fe-47a8-ae4c-b5797bf36489 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c40d925187c43e996a4bd03dfd4fde6'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:22:39.782998 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-d5f8018c-d2fe-47a8-ae4c-b5797bf36489 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c40d925187c43e996a4bd03dfd4fde6'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:22:39.789602 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-d5f8018c-d2fe-47a8-ae4c-b5797bf36489 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c40d925187c43e996a4bd03dfd4fde6'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:22:39.790628 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-d5f8018c-d2fe-47a8-ae4c-b5797bf36489 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c40d925187c43e996a4bd03dfd4fde6'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:22:39.800554 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-d5f8018c-d2fe-47a8-ae4c-b5797bf36489 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c40d925187c43e996a4bd03dfd4fde6'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:22:39.823150 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1186/2391] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:22:39 2022] DELETE /identity/v3/users/1c40d925187c43e996a4bd03dfd4fde6 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:22:39.828446 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74ffead-cced-4a3d-b1b5-dd6c1c173b14 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:22:39.828967 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74ffead-cced-4a3d-b1b5-dd6c1c173b14 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:22:39.829289 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74ffead-cced-4a3d-b1b5-dd6c1c173b14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:22:39.851493 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-e74ffead-cced-4a3d-b1b5-dd6c1c173b14 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:22:39.852845 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-e74ffead-cced-4a3d-b1b5-dd6c1c173b14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:22:39.972740 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1206/2392] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:22:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:22:39.984554 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f8018c-d2fe-47a8-ae4c-b5797bf36489 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:22:39.986499 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-d5f8018c-d2fe-47a8-ae4c-b5797bf36489 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:22:40.064518 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f8018c-d2fe-47a8-ae4c-b5797bf36489 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:22:40.068991 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15365401-8d39-4d35-b638-948a79e14e43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:22:40.069562 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15365401-8d39-4d35-b638-948a79e14e43 service neutron] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:22:40.069899 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15365401-8d39-4d35-b638-948a79e14e43 service neutron] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:22:40.070254 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15365401-8d39-4d35-b638-948a79e14e43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:22:40.082626 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15365401-8d39-4d35-b638-948a79e14e43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:22:40.084126 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15365401-8d39-4d35-b638-948a79e14e43 service neutron] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:22:40.097714 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1187/2393] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:22:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:22:40.535265 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74ffead-cced-4a3d-b1b5-dd6c1c173b14 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:22:40.538267 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-e74ffead-cced-4a3d-b1b5-dd6c1c173b14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:22:40.563517 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74ffead-cced-4a3d-b1b5-dd6c1c173b14 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:22:40.571604 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4814b5-a3f6-47fe-9cc7-631b130050d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:22:40.572490 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4814b5-a3f6-47fe-9cc7-631b130050d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:22:40.572820 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4814b5-a3f6-47fe-9cc7-631b130050d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:22:40.573187 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4814b5-a3f6-47fe-9cc7-631b130050d8 admin admin] PATH_INFO: `/v3/projects/b1b4a49134f34659af42d3bc351e6fd8` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:22:40.574664 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4814b5-a3f6-47fe-9cc7-631b130050d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1b4a49134f34659af42d3bc351e6fd8)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:22:40.577200 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4814b5-a3f6-47fe-9cc7-631b130050d8 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:22:40.602871 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:22:40.688049 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-1a4814b5-a3f6-47fe-9cc7-631b130050d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1b4a49134f34659af42d3bc351e6fd8', 'request_id': 'req-1a4814b5-a3f6-47fe-9cc7-631b130050d8'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:22:40.706312 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-1a4814b5-a3f6-47fe-9cc7-631b130050d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1b4a49134f34659af42d3bc351e6fd8', 'request_id': 'req-1a4814b5-a3f6-47fe-9cc7-631b130050d8'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:22:40.731429 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-1a4814b5-a3f6-47fe-9cc7-631b130050d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1b4a49134f34659af42d3bc351e6fd8', 'request_id': 'req-1a4814b5-a3f6-47fe-9cc7-631b130050d8'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:22:40.763860 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-1a4814b5-a3f6-47fe-9cc7-631b130050d8 admin admin] The token cache is being invalidate because project b1b4a49134f34659af42d3bc351e6fd8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81772) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:22:40.764551 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-1a4814b5-a3f6-47fe-9cc7-631b130050d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:22:40.766714 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-1a4814b5-a3f6-47fe-9cc7-631b130050d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:22:40.770162 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1207/2394] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:22:40 2022] DELETE /identity/v3/projects/b1b4a49134f34659af42d3bc351e6fd8 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:22:40.775903 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15365401-8d39-4d35-b638-948a79e14e43 service neutron] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:22:40.797002 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15365401-8d39-4d35-b638-948a79e14e43 service neutron] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:22:40.927432 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f753d920-83bd-42d6-b8e0-5ed472ebd01f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:22:40.928575 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f753d920-83bd-42d6-b8e0-5ed472ebd01f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:22:40.929102 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f753d920-83bd-42d6-b8e0-5ed472ebd01f admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:22:40.929603 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f753d920-83bd-42d6-b8e0-5ed472ebd01f admin admin] PATH_INFO: `/v3/users/8cd6045b5f5d4e43b752e703a3fc6cea` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:22:40.932615 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f753d920-83bd-42d6-b8e0-5ed472ebd01f admin admin] RBAC: Authorizing `identity:delete_user(user_id=8cd6045b5f5d4e43b752e703a3fc6cea)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:22:40.935392 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f753d920-83bd-42d6-b8e0-5ed472ebd01f admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:22:41.013250 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:22:41.058875 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-f753d920-83bd-42d6-b8e0-5ed472ebd01f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8cd6045b5f5d4e43b752e703a3fc6cea'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:22:41.070672 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-f753d920-83bd-42d6-b8e0-5ed472ebd01f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8cd6045b5f5d4e43b752e703a3fc6cea'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:22:41.071655 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-f753d920-83bd-42d6-b8e0-5ed472ebd01f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8cd6045b5f5d4e43b752e703a3fc6cea'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:22:41.072432 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-f753d920-83bd-42d6-b8e0-5ed472ebd01f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8cd6045b5f5d4e43b752e703a3fc6cea'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:22:41.082290 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-f753d920-83bd-42d6-b8e0-5ed472ebd01f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8cd6045b5f5d4e43b752e703a3fc6cea'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:22:41.104768 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1188/2395] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:22:40 2022] DELETE /identity/v3/users/8cd6045b5f5d4e43b752e703a3fc6cea => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:22:41.442247 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4814b5-a3f6-47fe-9cc7-631b130050d8 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:22:41.444820 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-1a4814b5-a3f6-47fe-9cc7-631b130050d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:22:41.470814 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4814b5-a3f6-47fe-9cc7-631b130050d8 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:22:41.588875 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5c521a-a66e-4a96-87da-f2b875c1a7a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:22:41.590063 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5c521a-a66e-4a96-87da-f2b875c1a7a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:22:41.590515 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5c521a-a66e-4a96-87da-f2b875c1a7a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:22:41.591022 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5c521a-a66e-4a96-87da-f2b875c1a7a8 admin admin] PATH_INFO: `/v3/projects/faa571e2bf5a457eabb0ea94032a5850` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:22:41.593125 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5c521a-a66e-4a96-87da-f2b875c1a7a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=faa571e2bf5a457eabb0ea94032a5850)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:22:41.596575 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5c521a-a66e-4a96-87da-f2b875c1a7a8 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:22:41.674257 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-4a5c521a-a66e-4a96-87da-f2b875c1a7a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'faa571e2bf5a457eabb0ea94032a5850', 'request_id': 'req-4a5c521a-a66e-4a96-87da-f2b875c1a7a8'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:22:41.686625 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-4a5c521a-a66e-4a96-87da-f2b875c1a7a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'faa571e2bf5a457eabb0ea94032a5850', 'request_id': 'req-4a5c521a-a66e-4a96-87da-f2b875c1a7a8'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:22:41.695635 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-4a5c521a-a66e-4a96-87da-f2b875c1a7a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'faa571e2bf5a457eabb0ea94032a5850', 'request_id': 'req-4a5c521a-a66e-4a96-87da-f2b875c1a7a8'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:22:41.710144 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-4a5c521a-a66e-4a96-87da-f2b875c1a7a8 admin admin] The token cache is being invalidate because project faa571e2bf5a457eabb0ea94032a5850 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81772) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:22:41.710590 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-4a5c521a-a66e-4a96-87da-f2b875c1a7a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:22:41.711301 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-4a5c521a-a66e-4a96-87da-f2b875c1a7a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:22:41.713208 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1208/2396] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:22:41 2022] DELETE /identity/v3/projects/faa571e2bf5a457eabb0ea94032a5850 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:22:41.718220 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f753d920-83bd-42d6-b8e0-5ed472ebd01f admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:22:41.720682 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-f753d920-83bd-42d6-b8e0-5ed472ebd01f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:22:41.744049 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f753d920-83bd-42d6-b8e0-5ed472ebd01f admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:22:41.827814 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8e8c04-b5a6-4860-8c7e-2ee6902352ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:22:41.828942 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8e8c04-b5a6-4860-8c7e-2ee6902352ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:22:41.829414 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8e8c04-b5a6-4860-8c7e-2ee6902352ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:22:41.829945 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8e8c04-b5a6-4860-8c7e-2ee6902352ae admin admin] PATH_INFO: `/v3/users/e0a52b8da9624be8ab62dea6fee92aff` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:22:41.832209 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8e8c04-b5a6-4860-8c7e-2ee6902352ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0a52b8da9624be8ab62dea6fee92aff)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:22:41.836364 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8e8c04-b5a6-4860-8c7e-2ee6902352ae admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:22:41.927294 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-0e8e8c04-b5a6-4860-8c7e-2ee6902352ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0a52b8da9624be8ab62dea6fee92aff'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:22:41.941894 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-0e8e8c04-b5a6-4860-8c7e-2ee6902352ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0a52b8da9624be8ab62dea6fee92aff'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:22:41.943287 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-0e8e8c04-b5a6-4860-8c7e-2ee6902352ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0a52b8da9624be8ab62dea6fee92aff'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:22:41.944708 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-0e8e8c04-b5a6-4860-8c7e-2ee6902352ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0a52b8da9624be8ab62dea6fee92aff'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:22:41.960395 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-0e8e8c04-b5a6-4860-8c7e-2ee6902352ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0a52b8da9624be8ab62dea6fee92aff'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:22:41.996952 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1189/2397] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:22:41 2022] DELETE /identity/v3/users/e0a52b8da9624be8ab62dea6fee92aff => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:22:42.465452 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1f7768-ac92-47da-8ee3-e15d1e269f3a None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:22:42.466233 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1f7768-ac92-47da-8ee3-e15d1e269f3a None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:22:42.466335 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1f7768-ac92-47da-8ee3-e15d1e269f3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:22:42.490195 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8e8c04-b5a6-4860-8c7e-2ee6902352ae admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:22:42.491713 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-0e8e8c04-b5a6-4860-8c7e-2ee6902352ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:22:42.494237 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-0b1f7768-ac92-47da-8ee3-e15d1e269f3a None None] MFA Rules not processed for user `1a52117df70447dba9bbee35d083e44f`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:22:42.497455 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-0b1f7768-ac92-47da-8ee3-e15d1e269f3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:22:42.540428 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:22:42.566651 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:22:42.609530 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1209/2398] 10.222.0.20 () {58 vars in 1009 bytes} [Tue Sep 6 20:22:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:22:42.623833 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8e8c04-b5a6-4860-8c7e-2ee6902352ae admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:22:42.635477 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ba6e3b-fc39-44c1-8f30-6de860d948b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:22:42.637100 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ba6e3b-fc39-44c1-8f30-6de860d948b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:22:42.637591 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ba6e3b-fc39-44c1-8f30-6de860d948b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:22:42.638280 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ba6e3b-fc39-44c1-8f30-6de860d948b4 admin admin] PATH_INFO: `/v3/projects/d794ca28ed7a4627b87f7c5400ec998c` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:22:42.640708 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ba6e3b-fc39-44c1-8f30-6de860d948b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d794ca28ed7a4627b87f7c5400ec998c)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:22:42.643021 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ba6e3b-fc39-44c1-8f30-6de860d948b4 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:22:42.664091 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1f7768-ac92-47da-8ee3-e15d1e269f3a None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:22:42.665973 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-0b1f7768-ac92-47da-8ee3-e15d1e269f3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:22:42.714602 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-c3ba6e3b-fc39-44c1-8f30-6de860d948b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd794ca28ed7a4627b87f7c5400ec998c', 'request_id': 'req-c3ba6e3b-fc39-44c1-8f30-6de860d948b4'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:22:42.723193 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-c3ba6e3b-fc39-44c1-8f30-6de860d948b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd794ca28ed7a4627b87f7c5400ec998c', 'request_id': 'req-c3ba6e3b-fc39-44c1-8f30-6de860d948b4'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:22:42.734585 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-c3ba6e3b-fc39-44c1-8f30-6de860d948b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd794ca28ed7a4627b87f7c5400ec998c', 'request_id': 'req-c3ba6e3b-fc39-44c1-8f30-6de860d948b4'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:22:42.750887 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-c3ba6e3b-fc39-44c1-8f30-6de860d948b4 admin admin] The token cache is being invalidate because project d794ca28ed7a4627b87f7c5400ec998c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81771) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:22:42.751445 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-c3ba6e3b-fc39-44c1-8f30-6de860d948b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:22:42.752438 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-c3ba6e3b-fc39-44c1-8f30-6de860d948b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:22:42.754461 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1190/2399] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:22:42 2022] DELETE /identity/v3/projects/d794ca28ed7a4627b87f7c5400ec998c => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 20:22:42.760013 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aac08c-c30f-48ae-805e-b785313f380a None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:22:42.760463 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aac08c-c30f-48ae-805e-b785313f380a None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:22:42.760844 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aac08c-c30f-48ae-805e-b785313f380a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:22:42.761019 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1f7768-ac92-47da-8ee3-e15d1e269f3a None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:22:42.787140 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-a3aac08c-c30f-48ae-805e-b785313f380a None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:22:42.788695 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-a3aac08c-c30f-48ae-805e-b785313f380a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:22:42.851221 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90e47d9-8317-4836-9822-b8dfc5f17346 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b72516236da47e292d78de201b0552b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:22:42.851873 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90e47d9-8317-4836-9822-b8dfc5f17346 service swift] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:22:42.852190 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90e47d9-8317-4836-9822-b8dfc5f17346 service swift] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:22:42.852810 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90e47d9-8317-4836-9822-b8dfc5f17346 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:22:42.854711 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-c90e47d9-8317-4836-9822-b8dfc5f17346 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:22:42.869635 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1191/2400] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:22:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:22:42.878033 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7050e89e-9607-4507-afe4-562e4fa67a7c None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:22:42.878285 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7050e89e-9607-4507-afe4-562e4fa67a7c None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:22:42.878762 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7050e89e-9607-4507-afe4-562e4fa67a7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:22:42.902265 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-7050e89e-9607-4507-afe4-562e4fa67a7c None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:22:42.903871 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-7050e89e-9607-4507-afe4-562e4fa67a7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:22:42.922013 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1192/2401] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:22:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:22:42.927458 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7050e89e-9607-4507-afe4-562e4fa67a7c None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:22:42.948463 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7050e89e-9607-4507-afe4-562e4fa67a7c None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:22:42.954654 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90e47d9-8317-4836-9822-b8dfc5f17346 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:22:42.955240 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb0e73d-ad6e-4de0-a98e-c6f82b753ba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:22:42.955882 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb0e73d-ad6e-4de0-a98e-c6f82b753ba0 admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:22:42.956198 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb0e73d-ad6e-4de0-a98e-c6f82b753ba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:22:42.956472 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb0e73d-ad6e-4de0-a98e-c6f82b753ba0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:22:42.957145 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb0e73d-ad6e-4de0-a98e-c6f82b753ba0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:22:42.957452 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb0e73d-ad6e-4de0-a98e-c6f82b753ba0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:22:42.959038 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90e47d9-8317-4836-9822-b8dfc5f17346 service swift] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:22:42.959397 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb0e73d-ad6e-4de0-a98e-c6f82b753ba0 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:22:42.976142 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1210/2402] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Sep 6 20:22:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 06 20:22:42.981453 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1193/2403] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 6 20:22:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:22:42.986673 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90e47d9-8317-4836-9822-b8dfc5f17346 service swift] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:22:43.006000 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90e47d9-8317-4836-9822-b8dfc5f17346 service swift] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:22:43.014632 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74bcb1c-afbe-418d-b864-c35a46e4afb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:22:43.015766 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74bcb1c-afbe-418d-b864-c35a46e4afb4 admin admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:22:43.016398 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74bcb1c-afbe-418d-b864-c35a46e4afb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:22:43.016869 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74bcb1c-afbe-418d-b864-c35a46e4afb4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:22:43.018071 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74bcb1c-afbe-418d-b864-c35a46e4afb4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:22:43.024059 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74bcb1c-afbe-418d-b864-c35a46e4afb4 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:22:43.068890 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:22:43.082427 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1211/2404] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:22:42 2022] POST /identity/v3/projects => generated 375 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:22:43.090471 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb0e73d-ad6e-4de0-a98e-c6f82b753ba0 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:22:43.119524 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb0e73d-ad6e-4de0-a98e-c6f82b753ba0 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:22:43.189530 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:22:43.251078 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951c2b49-08d3-4ae0-854a-52e7629db5af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:22:43.252389 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951c2b49-08d3-4ae0-854a-52e7629db5af admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:22:43.253010 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951c2b49-08d3-4ae0-854a-52e7629db5af admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:22:43.253515 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951c2b49-08d3-4ae0-854a-52e7629db5af admin admin] PATH_INFO: `/v3/users` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:22:43.255255 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951c2b49-08d3-4ae0-854a-52e7629db5af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:22:43.259761 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951c2b49-08d3-4ae0-854a-52e7629db5af admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:22:43.334498 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1194/2405] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:22:43 2022] POST /identity/v3/users => generated 288 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:22:43.340170 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74bcb1c-afbe-418d-b864-c35a46e4afb4 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:22:43.354764 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74bcb1c-afbe-418d-b864-c35a46e4afb4 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:22:43.362347 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612e5b86-f421-423e-9428-6253b08e487c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:22:43.363136 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612e5b86-f421-423e-9428-6253b08e487c admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:22:43.363485 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612e5b86-f421-423e-9428-6253b08e487c admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:22:43.363836 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612e5b86-f421-423e-9428-6253b08e487c admin admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:22:43.364684 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612e5b86-f421-423e-9428-6253b08e487c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:22:43.367182 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612e5b86-f421-423e-9428-6253b08e487c admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:22:43.389877 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1212/2406] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:22:43 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 20:22:43.391926 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951c2b49-08d3-4ae0-854a-52e7629db5af admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:22:43.407005 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951c2b49-08d3-4ae0-854a-52e7629db5af admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:22:43.419429 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8946a7d8-e5d0-40b8-8dd6-804e1e736bc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:22:43.420571 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8946a7d8-e5d0-40b8-8dd6-804e1e736bc3 admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:22:43.421064 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8946a7d8-e5d0-40b8-8dd6-804e1e736bc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:22:43.421736 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8946a7d8-e5d0-40b8-8dd6-804e1e736bc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:22:43.422811 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8946a7d8-e5d0-40b8-8dd6-804e1e736bc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:22:43.424885 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8946a7d8-e5d0-40b8-8dd6-804e1e736bc3 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:22:43.443561 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1195/2407] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:22:43 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:22:43.444552 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612e5b86-f421-423e-9428-6253b08e487c admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:22:43.464519 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612e5b86-f421-423e-9428-6253b08e487c admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:22:43.475401 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491d6a11-916e-4d17-bf10-716b333491d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:22:43.476153 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491d6a11-916e-4d17-bf10-716b333491d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:22:43.476608 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491d6a11-916e-4d17-bf10-716b333491d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:22:43.476970 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491d6a11-916e-4d17-bf10-716b333491d4 admin admin] PATH_INFO: `/v3/projects/ed4697b9e0a940b284b392b3e93197c7/users/7f661aa1f00e4c7e916b2498505475dc/roles/4eea39e24b06431aa357053fe7907a3d` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:22:43.513308 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491d6a11-916e-4d17-bf10-716b333491d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed4697b9e0a940b284b392b3e93197c7, user_id=7f661aa1f00e4c7e916b2498505475dc, role_id=4eea39e24b06431aa357053fe7907a3d)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:22:43.523838 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491d6a11-916e-4d17-bf10-716b333491d4 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:22:43.573106 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1213/2408] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:22:43 2022] PUT /identity/v3/projects/ed4697b9e0a940b284b392b3e93197c7/users/7f661aa1f00e4c7e916b2498505475dc/roles/4eea39e24b06431aa357053fe7907a3d => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 20:22:43.582186 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ec76e9-017a-40f7-bf73-368df66e1512 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:22:43.582186 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ec76e9-017a-40f7-bf73-368df66e1512 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:22:43.582186 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ec76e9-017a-40f7-bf73-368df66e1512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:22:43.605089 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-59ec76e9-017a-40f7-bf73-368df66e1512 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:22:43.606708 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-59ec76e9-017a-40f7-bf73-368df66e1512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:22:43.694473 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1196/2409] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:22:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:22:43.701037 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491d6a11-916e-4d17-bf10-716b333491d4 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:22:43.703947 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-491d6a11-916e-4d17-bf10-716b333491d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:22:43.889920 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491d6a11-916e-4d17-bf10-716b333491d4 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:22:43.902645 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23892a0-7209-431a-a660-8aedc1c57c3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:22:43.903988 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23892a0-7209-431a-a660-8aedc1c57c3b service neutron] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:22:43.904283 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23892a0-7209-431a-a660-8aedc1c57c3b service neutron] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:22:43.904719 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23892a0-7209-431a-a660-8aedc1c57c3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:22:43.927236 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23892a0-7209-431a-a660-8aedc1c57c3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:22:43.930484 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23892a0-7209-431a-a660-8aedc1c57c3b service neutron] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:22:43.949181 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1214/2410] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:22:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:22:53.558671 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ad119a-9168-4101-87a1-f8ef7a8d1274 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:22:53.559487 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ad119a-9168-4101-87a1-f8ef7a8d1274 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:22:53.559632 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ad119a-9168-4101-87a1-f8ef7a8d1274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:22:53.638385 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-00ad119a-9168-4101-87a1-f8ef7a8d1274 None None] MFA Rules not processed for user `7f661aa1f00e4c7e916b2498505475dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:22:53.641689 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-00ad119a-9168-4101-87a1-f8ef7a8d1274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:22:53.874199 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1197/2411] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:22:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:22:53.882033 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23892a0-7209-431a-a660-8aedc1c57c3b service neutron] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:22:53.906425 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23892a0-7209-431a-a660-8aedc1c57c3b service neutron] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:22:53.914598 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e5dc14-9e00-46ef-b166-54eb2bf4ea65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:22:53.915515 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e5dc14-9e00-46ef-b166-54eb2bf4ea65 admin admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:22:53.919765 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e5dc14-9e00-46ef-b166-54eb2bf4ea65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:22:53.919765 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e5dc14-9e00-46ef-b166-54eb2bf4ea65 admin admin] PATH_INFO: `/v3/projects` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:22:53.919765 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e5dc14-9e00-46ef-b166-54eb2bf4ea65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:22:53.919765 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e5dc14-9e00-46ef-b166-54eb2bf4ea65 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:22:54.002395 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1215/2412] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:22:53 2022] POST /identity/v3/projects => generated 379 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:22:54.003580 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ad119a-9168-4101-87a1-f8ef7a8d1274 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:22:54.032526 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ad119a-9168-4101-87a1-f8ef7a8d1274 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:22:54.161271 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:22:54.196383 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0181eb7e-259a-421f-81d7-25fb09b38cb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:22:54.197592 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0181eb7e-259a-421f-81d7-25fb09b38cb7 admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:22:54.198510 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0181eb7e-259a-421f-81d7-25fb09b38cb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:22:54.199103 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0181eb7e-259a-421f-81d7-25fb09b38cb7 admin admin] PATH_INFO: `/v3/users` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:22:54.200327 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0181eb7e-259a-421f-81d7-25fb09b38cb7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:22:54.205726 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0181eb7e-259a-421f-81d7-25fb09b38cb7 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:22:54.242023 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1198/2413] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:22:54 2022] POST /identity/v3/users => generated 296 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:22:54.244295 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e5dc14-9e00-46ef-b166-54eb2bf4ea65 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:22:54.310930 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e5dc14-9e00-46ef-b166-54eb2bf4ea65 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:22:54.330054 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ea9ece-6122-43af-88a2-a002666c38bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:22:54.330695 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ea9ece-6122-43af-88a2-a002666c38bb admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:22:54.331050 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ea9ece-6122-43af-88a2-a002666c38bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:22:54.331336 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ea9ece-6122-43af-88a2-a002666c38bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:22:54.332357 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ea9ece-6122-43af-88a2-a002666c38bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:22:54.334261 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ea9ece-6122-43af-88a2-a002666c38bb admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:22:54.368283 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0181eb7e-259a-421f-81d7-25fb09b38cb7 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:22:54.368963 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1216/2414] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:22:54 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:22:54.394767 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0181eb7e-259a-421f-81d7-25fb09b38cb7 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:22:54.408360 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3ee563-dd2c-4442-91e6-79fc786e3c14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:22:54.409245 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3ee563-dd2c-4442-91e6-79fc786e3c14 admin admin] REQUEST_METHOD: `PUT` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:22:54.409760 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3ee563-dd2c-4442-91e6-79fc786e3c14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:22:54.410206 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3ee563-dd2c-4442-91e6-79fc786e3c14 admin admin] PATH_INFO: `/v3/projects/f3f976e76eb841ea871010a2eef1dc8f/users/b2219ed125b2449f8a738834af8808b5/roles/e0ab41d8ae22474688b5361bcc75e0f0` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:22:54.437875 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3ee563-dd2c-4442-91e6-79fc786e3c14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3f976e76eb841ea871010a2eef1dc8f, user_id=b2219ed125b2449f8a738834af8808b5, role_id=e0ab41d8ae22474688b5361bcc75e0f0)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:22:54.440603 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3ee563-dd2c-4442-91e6-79fc786e3c14 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:22:54.459267 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1199/2415] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:22:54 2022] PUT /identity/v3/projects/f3f976e76eb841ea871010a2eef1dc8f/users/b2219ed125b2449f8a738834af8808b5/roles/e0ab41d8ae22474688b5361bcc75e0f0 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:22:54.463693 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ea9ece-6122-43af-88a2-a002666c38bb admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:22:54.476264 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ea9ece-6122-43af-88a2-a002666c38bb admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:22:54.600844 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e824361-71db-49a7-9db9-c988acf59624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:22:54.601536 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e824361-71db-49a7-9db9-c988acf59624 admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:22:54.601952 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e824361-71db-49a7-9db9-c988acf59624 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:22:54.602236 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e824361-71db-49a7-9db9-c988acf59624 admin admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:22:54.602948 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e824361-71db-49a7-9db9-c988acf59624 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:22:54.605116 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e824361-71db-49a7-9db9-c988acf59624 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:22:54.621366 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1217/2416] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:22:54 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:22:54.627408 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3ee563-dd2c-4442-91e6-79fc786e3c14 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:22:54.641794 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3ee563-dd2c-4442-91e6-79fc786e3c14 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:22:54.649688 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ca0fbe-2e2b-4aa6-9659-8ccbac318d84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:22:54.650058 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ca0fbe-2e2b-4aa6-9659-8ccbac318d84 admin admin] REQUEST_METHOD: `PUT` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:22:54.650540 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ca0fbe-2e2b-4aa6-9659-8ccbac318d84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:22:54.651030 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ca0fbe-2e2b-4aa6-9659-8ccbac318d84 admin admin] PATH_INFO: `/v3/projects/f3f976e76eb841ea871010a2eef1dc8f/users/b2219ed125b2449f8a738834af8808b5/roles/e0ab41d8ae22474688b5361bcc75e0f0` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:22:54.654087 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ca0fbe-2e2b-4aa6-9659-8ccbac318d84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3f976e76eb841ea871010a2eef1dc8f, user_id=b2219ed125b2449f8a738834af8808b5, role_id=e0ab41d8ae22474688b5361bcc75e0f0)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:22:54.657344 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ca0fbe-2e2b-4aa6-9659-8ccbac318d84 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:22:54.675885 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1200/2417] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:22:54 2022] PUT /identity/v3/projects/f3f976e76eb841ea871010a2eef1dc8f/users/b2219ed125b2449f8a738834af8808b5/roles/e0ab41d8ae22474688b5361bcc75e0f0 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:23:04.614115 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=84,keystone:INSERT=2 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:23:04.670119 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=24,keystone:INSERT=5 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:23:07.759232 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2201e134-41a3-4fa2-b476-5fbcf027c514 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:07.760363 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2201e134-41a3-4fa2-b476-5fbcf027c514 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:07.760998 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2201e134-41a3-4fa2-b476-5fbcf027c514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:07.825812 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-2201e134-41a3-4fa2-b476-5fbcf027c514 None None] MFA Rules not processed for user `b2219ed125b2449f8a738834af8808b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:23:07.827659 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-2201e134-41a3-4fa2-b476-5fbcf027c514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:23:07.931307 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1218/2418] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:23:07 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:23:07.961907 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ca0fbe-2e2b-4aa6-9659-8ccbac318d84 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:23:07.963900 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-a2ca0fbe-2e2b-4aa6-9659-8ccbac318d84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:23:08.142615 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ca0fbe-2e2b-4aa6-9659-8ccbac318d84 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:23:08.152363 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c521ae6a-6196-4a09-af3e-e706ef8d2320 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5776cbf4f6714b2083c3ebdd6d98cc87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:23:08.153590 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c521ae6a-6196-4a09-af3e-e706ef8d2320 service nova] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:08.154091 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c521ae6a-6196-4a09-af3e-e706ef8d2320 service nova] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:08.154546 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c521ae6a-6196-4a09-af3e-e706ef8d2320 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:08.241069 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c521ae6a-6196-4a09-af3e-e706ef8d2320 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:23:08.244205 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c521ae6a-6196-4a09-af3e-e706ef8d2320 service nova] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:23:08.260731 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1201/2419] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:23:07 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 20:23:12.204619 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2201e134-41a3-4fa2-b476-5fbcf027c514 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:23:12.207920 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-2201e134-41a3-4fa2-b476-5fbcf027c514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:23:12.471128 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2201e134-41a3-4fa2-b476-5fbcf027c514 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:23:12.480106 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e043ac7-0afe-4c26-a66c-9ad85538220c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f57872de7bb74ea48d30e93e51830aba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:23:12.481086 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e043ac7-0afe-4c26-a66c-9ad85538220c service placement] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:12.481687 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e043ac7-0afe-4c26-a66c-9ad85538220c service placement] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:12.482167 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e043ac7-0afe-4c26-a66c-9ad85538220c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:12.484445 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-7e043ac7-0afe-4c26-a66c-9ad85538220c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:23:12.488951 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c521ae6a-6196-4a09-af3e-e706ef8d2320 service nova] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:23:12.496985 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-c521ae6a-6196-4a09-af3e-e706ef8d2320 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:23:12.511082 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e043ac7-0afe-4c26-a66c-9ad85538220c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:23:12.514599 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e043ac7-0afe-4c26-a66c-9ad85538220c service placement] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:23:12.536813 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c521ae6a-6196-4a09-af3e-e706ef8d2320 service nova] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:23:12.543078 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1219/2420] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:23:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:23:12.545165 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62297e6-4a39-428d-806b-5acb3277eacf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f57872de7bb74ea48d30e93e51830aba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:23:12.546168 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62297e6-4a39-428d-806b-5acb3277eacf service placement] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:12.546895 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62297e6-4a39-428d-806b-5acb3277eacf service placement] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:12.547375 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62297e6-4a39-428d-806b-5acb3277eacf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:12.559119 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62297e6-4a39-428d-806b-5acb3277eacf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:23:12.561088 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62297e6-4a39-428d-806b-5acb3277eacf service placement] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:23:12.582599 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1202/2421] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:23:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:23:15.921435 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c278a5-e46e-4387-8234-8bb90895a9ee None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:15.922344 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c278a5-e46e-4387-8234-8bb90895a9ee None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:15.922344 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c278a5-e46e-4387-8234-8bb90895a9ee None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:15.923770 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1220/2422] 10.222.0.20 () {54 vars in 921 bytes} [Tue Sep 6 20:23:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:23:15.944387 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b884b28-6a3e-4137-a9c8-47566cae1bfa None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:15.946141 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b884b28-6a3e-4137-a9c8-47566cae1bfa None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:15.947160 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b884b28-6a3e-4137-a9c8-47566cae1bfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:16.000641 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-5b884b28-6a3e-4137-a9c8-47566cae1bfa None None] MFA Rules not processed for user `5776cbf4f6714b2083c3ebdd6d98cc87`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:23:16.009720 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-5b884b28-6a3e-4137-a9c8-47566cae1bfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:23:16.043404 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1203/2423] 10.222.0.20 () {58 vars in 1018 bytes} [Tue Sep 6 20:23:15 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:23:16.075109 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c278a5-e46e-4387-8234-8bb90895a9ee None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:23:16.079097 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-c3c278a5-e46e-4387-8234-8bb90895a9ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:23:16.367988 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c278a5-e46e-4387-8234-8bb90895a9ee None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:23:16.385269 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cf130e-c90e-4395-a66f-9e1039d28a35 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07af18f35ffc42a295278d5349e06381', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:23:16.386111 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cf130e-c90e-4395-a66f-9e1039d28a35 service glance] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:16.387267 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cf130e-c90e-4395-a66f-9e1039d28a35 service glance] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:16.387267 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cf130e-c90e-4395-a66f-9e1039d28a35 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:16.426328 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cf130e-c90e-4395-a66f-9e1039d28a35 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:23:16.430865 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cf130e-c90e-4395-a66f-9e1039d28a35 service glance] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:23:16.443252 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:23:16.467245 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1221/2424] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:23:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:23:16.761204 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c150b3-f203-45e7-824a-5636480b5612 None None] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:16.762162 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c150b3-f203-45e7-824a-5636480b5612 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:16.762632 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c150b3-f203-45e7-824a-5636480b5612 None None] PATH_INFO: `/` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:16.764986 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1204/2425] 10.222.0.20 () {54 vars in 921 bytes} [Tue Sep 6 20:23:16 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:23:16.778716 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e259f6-7389-41e1-a727-222912be1f6d None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:16.779275 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e259f6-7389-41e1-a727-222912be1f6d None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:16.786930 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e259f6-7389-41e1-a727-222912be1f6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:16.833960 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-a7e259f6-7389-41e1-a727-222912be1f6d None None] MFA Rules not processed for user `32882d38295046fea004a659a71bde58`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:23:16.840845 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-a7e259f6-7389-41e1-a727-222912be1f6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:23:16.979849 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1222/2426] 10.222.0.20 () {58 vars in 1018 bytes} [Tue Sep 6 20:23:16 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:23:16.998470 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c150b3-f203-45e7-824a-5636480b5612 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:23:17.004385 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-99c150b3-f203-45e7-824a-5636480b5612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:23:17.032024 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c150b3-f203-45e7-824a-5636480b5612 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:23:17.040819 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c65178e-6b73-4a1e-9969-1d580bdcd64c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:23:17.042335 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c65178e-6b73-4a1e-9969-1d580bdcd64c service neutron] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:17.043376 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c65178e-6b73-4a1e-9969-1d580bdcd64c service neutron] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:17.044162 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c65178e-6b73-4a1e-9969-1d580bdcd64c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:17.068054 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c65178e-6b73-4a1e-9969-1d580bdcd64c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:23:17.071078 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c65178e-6b73-4a1e-9969-1d580bdcd64c service neutron] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:23:17.087587 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1205/2427] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:23:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:23:17.216607 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c422cdb3-a6f0-4efd-bd88-34cd81da6390 None None] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:17.217402 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c422cdb3-a6f0-4efd-bd88-34cd81da6390 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:17.218031 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c422cdb3-a6f0-4efd-bd88-34cd81da6390 None None] PATH_INFO: `/` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:17.220327 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1223/2428] 10.222.0.20 () {54 vars in 921 bytes} [Tue Sep 6 20:23:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 20:23:17.231538 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842648ca-4e89-4d90-a198-9416ad1ebf02 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:17.231969 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842648ca-4e89-4d90-a198-9416ad1ebf02 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:17.232353 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842648ca-4e89-4d90-a198-9416ad1ebf02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:17.271245 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-842648ca-4e89-4d90-a198-9416ad1ebf02 None None] MFA Rules not processed for user `5776cbf4f6714b2083c3ebdd6d98cc87`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:23:17.274757 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-842648ca-4e89-4d90-a198-9416ad1ebf02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:23:17.319986 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1206/2429] 10.222.0.20 () {58 vars in 1018 bytes} [Tue Sep 6 20:23:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:23:17.337374 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c422cdb3-a6f0-4efd-bd88-34cd81da6390 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:23:17.354835 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c422cdb3-a6f0-4efd-bd88-34cd81da6390 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:23:17.359996 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce4b000-8562-4ccd-a00d-18d5395c4f2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:23:17.361214 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce4b000-8562-4ccd-a00d-18d5395c4f2e service neutron] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:17.361830 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce4b000-8562-4ccd-a00d-18d5395c4f2e service neutron] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:17.362668 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce4b000-8562-4ccd-a00d-18d5395c4f2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:17.402737 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce4b000-8562-4ccd-a00d-18d5395c4f2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:23:17.404343 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce4b000-8562-4ccd-a00d-18d5395c4f2e service neutron] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:23:17.421785 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1224/2430] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:23:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:23:20.686169 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7399a7f7-da69-4779-8a4d-5f0548a035ba None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:20.686797 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7399a7f7-da69-4779-8a4d-5f0548a035ba None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:20.686797 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7399a7f7-da69-4779-8a4d-5f0548a035ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:20.708820 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-7399a7f7-da69-4779-8a4d-5f0548a035ba None None] MFA Rules not processed for user `1a52117df70447dba9bbee35d083e44f`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:23:20.710270 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-7399a7f7-da69-4779-8a4d-5f0548a035ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:23:20.742464 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:23:20.777007 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1207/2431] 10.222.0.20 () {58 vars in 1009 bytes} [Tue Sep 6 20:23:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:23:20.793882 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce4b000-8562-4ccd-a00d-18d5395c4f2e service neutron] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:23:20.796117 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-bce4b000-8562-4ccd-a00d-18d5395c4f2e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:23:20.886672 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce4b000-8562-4ccd-a00d-18d5395c4f2e service neutron] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:23:20.892761 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6e4f4f-f6c0-4cdd-8935-e89f97ddbb51 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b72516236da47e292d78de201b0552b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:23:20.893564 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6e4f4f-f6c0-4cdd-8935-e89f97ddbb51 service swift] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:20.893906 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6e4f4f-f6c0-4cdd-8935-e89f97ddbb51 service swift] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:20.894288 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6e4f4f-f6c0-4cdd-8935-e89f97ddbb51 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:20.907715 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6e4f4f-f6c0-4cdd-8935-e89f97ddbb51 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:23:20.909731 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6e4f4f-f6c0-4cdd-8935-e89f97ddbb51 service swift] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:23:20.921115 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1225/2432] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Sep 6 20:23:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 20:23:21.117854 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7399a7f7-da69-4779-8a4d-5f0548a035ba None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:23:21.135315 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7399a7f7-da69-4779-8a4d-5f0548a035ba None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:23:21.144424 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e6fe4b-d9dd-41ca-83b0-e49bd9fa20f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:23:21.145437 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e6fe4b-d9dd-41ca-83b0-e49bd9fa20f3 service neutron] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:21.145795 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e6fe4b-d9dd-41ca-83b0-e49bd9fa20f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:21.146041 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e6fe4b-d9dd-41ca-83b0-e49bd9fa20f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:21.150752 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-46e6fe4b-d9dd-41ca-83b0-e49bd9fa20f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:23:21.265643 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e6fe4b-d9dd-41ca-83b0-e49bd9fa20f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:23:21.267279 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e6fe4b-d9dd-41ca-83b0-e49bd9fa20f3 service neutron] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:23:21.285514 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1208/2433] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:23:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:23:28.318958 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6e4f4f-f6c0-4cdd-8935-e89f97ddbb51 service swift] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:23:28.356862 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6e4f4f-f6c0-4cdd-8935-e89f97ddbb51 service swift] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:23:28.373323 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5554cb24-28ec-48f8-8663-a7e83e624be3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f57872de7bb74ea48d30e93e51830aba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:23:28.374230 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5554cb24-28ec-48f8-8663-a7e83e624be3 service placement] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:28.374746 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5554cb24-28ec-48f8-8663-a7e83e624be3 service placement] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:28.374969 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5554cb24-28ec-48f8-8663-a7e83e624be3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:28.377579 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-5554cb24-28ec-48f8-8663-a7e83e624be3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:23:28.395543 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e6fe4b-d9dd-41ca-83b0-e49bd9fa20f3 service neutron] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:23:28.398686 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-46e6fe4b-d9dd-41ca-83b0-e49bd9fa20f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:23:28.422667 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5554cb24-28ec-48f8-8663-a7e83e624be3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:23:28.424941 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5554cb24-28ec-48f8-8663-a7e83e624be3 service placement] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:23:28.437513 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e6fe4b-d9dd-41ca-83b0-e49bd9fa20f3 service neutron] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:23:28.439748 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1226/2434] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:23:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:23:28.443936 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e09600f-bea2-4baa-83e2-1ccb238e5f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:23:28.444811 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e09600f-bea2-4baa-83e2-1ccb238e5f60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:28.445261 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e09600f-bea2-4baa-83e2-1ccb238e5f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:28.445811 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e09600f-bea2-4baa-83e2-1ccb238e5f60 admin admin] PATH_INFO: `/v3/users/f49ef54a38614cb880577626e65b0e32` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:28.447463 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e09600f-bea2-4baa-83e2-1ccb238e5f60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f49ef54a38614cb880577626e65b0e32)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:23:28.450996 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e09600f-bea2-4baa-83e2-1ccb238e5f60 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:23:28.583003 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-7e09600f-bea2-4baa-83e2-1ccb238e5f60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f49ef54a38614cb880577626e65b0e32'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:23:28.601117 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-7e09600f-bea2-4baa-83e2-1ccb238e5f60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f49ef54a38614cb880577626e65b0e32'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:23:28.601117 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-7e09600f-bea2-4baa-83e2-1ccb238e5f60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f49ef54a38614cb880577626e65b0e32'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:23:28.601721 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-7e09600f-bea2-4baa-83e2-1ccb238e5f60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f49ef54a38614cb880577626e65b0e32'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:23:28.619321 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-7e09600f-bea2-4baa-83e2-1ccb238e5f60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f49ef54a38614cb880577626e65b0e32'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:23:28.674664 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1209/2435] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:23:28 2022] DELETE /identity/v3/users/f49ef54a38614cb880577626e65b0e32 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:23:28.957737 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5554cb24-28ec-48f8-8663-a7e83e624be3 service placement] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:23:28.983672 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5554cb24-28ec-48f8-8663-a7e83e624be3 service placement] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:23:29.183328 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4ed72b-cea2-40ee-924e-128d054a60d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:23:29.184404 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4ed72b-cea2-40ee-924e-128d054a60d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:29.184870 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4ed72b-cea2-40ee-924e-128d054a60d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:29.185356 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4ed72b-cea2-40ee-924e-128d054a60d7 admin admin] PATH_INFO: `/v3/projects/27f27bdf0ac14f62b682027b05b219ee` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:29.187928 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4ed72b-cea2-40ee-924e-128d054a60d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=27f27bdf0ac14f62b682027b05b219ee)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:23:29.192411 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4ed72b-cea2-40ee-924e-128d054a60d7 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:23:29.219687 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:23:29.356665 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-4b4ed72b-cea2-40ee-924e-128d054a60d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27f27bdf0ac14f62b682027b05b219ee', 'request_id': 'req-4b4ed72b-cea2-40ee-924e-128d054a60d7'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:23:29.375557 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-4b4ed72b-cea2-40ee-924e-128d054a60d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27f27bdf0ac14f62b682027b05b219ee', 'request_id': 'req-4b4ed72b-cea2-40ee-924e-128d054a60d7'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:23:29.394875 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-4b4ed72b-cea2-40ee-924e-128d054a60d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27f27bdf0ac14f62b682027b05b219ee', 'request_id': 'req-4b4ed72b-cea2-40ee-924e-128d054a60d7'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:23:29.422996 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-4b4ed72b-cea2-40ee-924e-128d054a60d7 admin admin] The token cache is being invalidate because project 27f27bdf0ac14f62b682027b05b219ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81772) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:23:29.423519 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-4b4ed72b-cea2-40ee-924e-128d054a60d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:23:29.425229 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-4b4ed72b-cea2-40ee-924e-128d054a60d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:23:29.429246 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1227/2436] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:23:28 2022] DELETE /identity/v3/projects/27f27bdf0ac14f62b682027b05b219ee => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 20:23:29.438125 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b16406-3ed5-4b79-b870-df8e3e3c17a2 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:29.438871 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b16406-3ed5-4b79-b870-df8e3e3c17a2 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:29.439334 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b16406-3ed5-4b79-b870-df8e3e3c17a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:29.477403 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-62b16406-3ed5-4b79-b870-df8e3e3c17a2 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:23:29.479914 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-62b16406-3ed5-4b79-b870-df8e3e3c17a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:23:29.583281 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:23:29.641436 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1210/2437] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:23:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:23:29.656772 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe52baf0-e092-41c6-8f85-6e3aac3971ab None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:29.657466 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe52baf0-e092-41c6-8f85-6e3aac3971ab None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:29.658064 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe52baf0-e092-41c6-8f85-6e3aac3971ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:29.694660 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-fe52baf0-e092-41c6-8f85-6e3aac3971ab None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:23:29.697087 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-fe52baf0-e092-41c6-8f85-6e3aac3971ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:23:29.711852 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1228/2438] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:23:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:23:29.717358 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b16406-3ed5-4b79-b870-df8e3e3c17a2 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:23:29.733141 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b16406-3ed5-4b79-b870-df8e3e3c17a2 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:23:29.739104 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d2196c-5aad-4bfb-a3c9-46f6b22aa881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:23:29.740004 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d2196c-5aad-4bfb-a3c9-46f6b22aa881 admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:29.740656 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d2196c-5aad-4bfb-a3c9-46f6b22aa881 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:29.740856 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d2196c-5aad-4bfb-a3c9-46f6b22aa881 admin admin] PATH_INFO: `/v3/domains` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:29.741559 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d2196c-5aad-4bfb-a3c9-46f6b22aa881 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:23:29.741971 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d2196c-5aad-4bfb-a3c9-46f6b22aa881 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:23:29.743931 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d2196c-5aad-4bfb-a3c9-46f6b22aa881 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:23:29.760168 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1211/2439] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 6 20:23:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:23:29.769105 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe52baf0-e092-41c6-8f85-6e3aac3971ab None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:23:29.787599 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe52baf0-e092-41c6-8f85-6e3aac3971ab None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:23:29.796283 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5847ed2b-f339-4da8-b121-b667899142af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:23:29.797340 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5847ed2b-f339-4da8-b121-b667899142af admin admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:29.797868 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5847ed2b-f339-4da8-b121-b667899142af admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:29.798360 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5847ed2b-f339-4da8-b121-b667899142af admin admin] PATH_INFO: `/v3/projects` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:29.799431 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5847ed2b-f339-4da8-b121-b667899142af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:23:29.803382 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5847ed2b-f339-4da8-b121-b667899142af admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:23:29.859071 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1229/2440] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:23:29 2022] POST /identity/v3/projects => generated 367 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:23:29.869287 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d2196c-5aad-4bfb-a3c9-46f6b22aa881 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:23:29.891572 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d2196c-5aad-4bfb-a3c9-46f6b22aa881 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:23:30.035068 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1943fc-b360-428d-941b-f14da3f5f1a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:23:30.036156 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1943fc-b360-428d-941b-f14da3f5f1a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:30.036760 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1943fc-b360-428d-941b-f14da3f5f1a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:30.037240 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1943fc-b360-428d-941b-f14da3f5f1a1 admin admin] PATH_INFO: `/v3/users` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:30.038533 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1943fc-b360-428d-941b-f14da3f5f1a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:23:30.043013 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1943fc-b360-428d-941b-f14da3f5f1a1 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:23:30.111632 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5847ed2b-f339-4da8-b121-b667899142af admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:23:30.122481 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1212/2441] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:23:29 2022] POST /identity/v3/users => generated 284 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 20:23:30.155082 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5847ed2b-f339-4da8-b121-b667899142af admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:23:30.224197 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569014d9-dfac-407c-ad36-9cc610253711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:23:30.226245 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569014d9-dfac-407c-ad36-9cc610253711 admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:30.226940 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569014d9-dfac-407c-ad36-9cc610253711 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:30.227594 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569014d9-dfac-407c-ad36-9cc610253711 admin admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:30.229055 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569014d9-dfac-407c-ad36-9cc610253711 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:23:30.232272 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569014d9-dfac-407c-ad36-9cc610253711 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:23:30.265854 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1230/2442] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:23:30 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:23:30.267301 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1943fc-b360-428d-941b-f14da3f5f1a1 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:23:30.331668 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1943fc-b360-428d-941b-f14da3f5f1a1 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:23:30.342096 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c88dc6-ac1a-45ab-b36b-207ee964d782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:23:30.342096 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c88dc6-ac1a-45ab-b36b-207ee964d782 admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:30.342096 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c88dc6-ac1a-45ab-b36b-207ee964d782 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:30.342096 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c88dc6-ac1a-45ab-b36b-207ee964d782 admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:30.354522 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c88dc6-ac1a-45ab-b36b-207ee964d782 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:23:30.356479 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c88dc6-ac1a-45ab-b36b-207ee964d782 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:23:30.379962 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569014d9-dfac-407c-ad36-9cc610253711 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:23:30.384622 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1213/2443] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:23:30 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:23:30.402773 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569014d9-dfac-407c-ad36-9cc610253711 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:23:30.411342 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac76454-2d57-4953-88d8-2e3304d15983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:23:30.413055 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac76454-2d57-4953-88d8-2e3304d15983 admin admin] REQUEST_METHOD: `PUT` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:30.413351 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac76454-2d57-4953-88d8-2e3304d15983 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:30.413734 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac76454-2d57-4953-88d8-2e3304d15983 admin admin] PATH_INFO: `/v3/projects/019ed76a1237479fb4a69e5dec27cbde/users/9f89c2b4302047479f3d2abd35b53aac/roles/4eea39e24b06431aa357053fe7907a3d` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:30.464266 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac76454-2d57-4953-88d8-2e3304d15983 admin admin] RBAC: Authorizing `identity:create_grant(project_id=019ed76a1237479fb4a69e5dec27cbde, user_id=9f89c2b4302047479f3d2abd35b53aac, role_id=4eea39e24b06431aa357053fe7907a3d)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:23:30.469015 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac76454-2d57-4953-88d8-2e3304d15983 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:23:30.490280 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1231/2444] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:23:30 2022] PUT /identity/v3/projects/019ed76a1237479fb4a69e5dec27cbde/users/9f89c2b4302047479f3d2abd35b53aac/roles/4eea39e24b06431aa357053fe7907a3d => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:23:30.509735 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69306953-03b7-4629-bc6b-b9a15f28cbfd None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:30.510506 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69306953-03b7-4629-bc6b-b9a15f28cbfd None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:30.510506 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69306953-03b7-4629-bc6b-b9a15f28cbfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:30.563629 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-69306953-03b7-4629-bc6b-b9a15f28cbfd None None] MFA Rules not processed for user `9f89c2b4302047479f3d2abd35b53aac`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:23:30.565887 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-69306953-03b7-4629-bc6b-b9a15f28cbfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:23:30.697630 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1214/2445] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:23:30 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:23:30.702791 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac76454-2d57-4953-88d8-2e3304d15983 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:23:30.776853 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac76454-2d57-4953-88d8-2e3304d15983 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:23:30.782561 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13c5c99-2e72-43c6-bbe7-f9025d818e65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:23:30.783346 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13c5c99-2e72-43c6-bbe7-f9025d818e65 admin admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:30.783726 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13c5c99-2e72-43c6-bbe7-f9025d818e65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:30.784011 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13c5c99-2e72-43c6-bbe7-f9025d818e65 admin admin] PATH_INFO: `/v3/projects` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:30.784713 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13c5c99-2e72-43c6-bbe7-f9025d818e65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:23:30.786804 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13c5c99-2e72-43c6-bbe7-f9025d818e65 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:23:30.822319 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1232/2446] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:23:30 2022] POST /identity/v3/projects => generated 365 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:23:30.828342 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69306953-03b7-4629-bc6b-b9a15f28cbfd None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:23:30.840513 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69306953-03b7-4629-bc6b-b9a15f28cbfd None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:23:30.901084 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:23:30.941405 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e60df62-a492-47ad-8e1b-4daa3f96f426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:23:30.942395 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e60df62-a492-47ad-8e1b-4daa3f96f426 admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:30.942852 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e60df62-a492-47ad-8e1b-4daa3f96f426 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:30.943204 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e60df62-a492-47ad-8e1b-4daa3f96f426 admin admin] PATH_INFO: `/v3/users` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:30.944197 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e60df62-a492-47ad-8e1b-4daa3f96f426 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:23:30.947748 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e60df62-a492-47ad-8e1b-4daa3f96f426 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:23:30.985011 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1215/2447] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:23:30 2022] POST /identity/v3/users => generated 289 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:23:30.986876 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13c5c99-2e72-43c6-bbe7-f9025d818e65 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:23:31.001976 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13c5c99-2e72-43c6-bbe7-f9025d818e65 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:23:31.008064 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d574f0-25b8-46f4-8ce7-c3c97e4cc1ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:23:31.008699 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d574f0-25b8-46f4-8ce7-c3c97e4cc1ce admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:31.008987 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d574f0-25b8-46f4-8ce7-c3c97e4cc1ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:31.009263 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d574f0-25b8-46f4-8ce7-c3c97e4cc1ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:31.009963 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d574f0-25b8-46f4-8ce7-c3c97e4cc1ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:23:31.011752 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d574f0-25b8-46f4-8ce7-c3c97e4cc1ce admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:23:31.024406 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1233/2448] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:23:30 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 20:23:31.029471 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e60df62-a492-47ad-8e1b-4daa3f96f426 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:23:31.041577 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e60df62-a492-47ad-8e1b-4daa3f96f426 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:23:31.046866 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1194570-4948-4651-b14b-ac74ceaee212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:23:31.047558 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1194570-4948-4651-b14b-ac74ceaee212 admin admin] REQUEST_METHOD: `PUT` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:31.047838 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1194570-4948-4651-b14b-ac74ceaee212 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:31.048112 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1194570-4948-4651-b14b-ac74ceaee212 admin admin] PATH_INFO: `/v3/projects/5a5145bbe4f7423d963ad555fee26090/users/242b7310e86d4687b08d680fcc824d99/roles/e0ab41d8ae22474688b5361bcc75e0f0` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:31.072999 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1194570-4948-4651-b14b-ac74ceaee212 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a5145bbe4f7423d963ad555fee26090, user_id=242b7310e86d4687b08d680fcc824d99, role_id=e0ab41d8ae22474688b5361bcc75e0f0)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:23:31.076436 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1194570-4948-4651-b14b-ac74ceaee212 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:23:31.095409 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1216/2449] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:23:31 2022] PUT /identity/v3/projects/5a5145bbe4f7423d963ad555fee26090/users/242b7310e86d4687b08d680fcc824d99/roles/e0ab41d8ae22474688b5361bcc75e0f0 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:23:31.100185 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d574f0-25b8-46f4-8ce7-c3c97e4cc1ce admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:23:31.112134 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d574f0-25b8-46f4-8ce7-c3c97e4cc1ce admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:23:31.122435 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:23:31.194602 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51fab3c-c2eb-4ca0-b370-9e74e1caebce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:23:31.194989 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51fab3c-c2eb-4ca0-b370-9e74e1caebce admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:31.195326 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51fab3c-c2eb-4ca0-b370-9e74e1caebce admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:31.195584 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51fab3c-c2eb-4ca0-b370-9e74e1caebce admin admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:31.196268 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51fab3c-c2eb-4ca0-b370-9e74e1caebce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:23:31.198140 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51fab3c-c2eb-4ca0-b370-9e74e1caebce admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:23:31.210440 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1234/2450] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:23:31 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:23:31.215530 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1194570-4948-4651-b14b-ac74ceaee212 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:23:31.232594 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1194570-4948-4651-b14b-ac74ceaee212 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:23:31.241322 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffb29a6-ddd8-4576-b3e0-ab7d50152e04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:23:31.242778 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffb29a6-ddd8-4576-b3e0-ab7d50152e04 admin admin] REQUEST_METHOD: `PUT` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:31.243329 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffb29a6-ddd8-4576-b3e0-ab7d50152e04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:31.243862 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffb29a6-ddd8-4576-b3e0-ab7d50152e04 admin admin] PATH_INFO: `/v3/projects/5a5145bbe4f7423d963ad555fee26090/users/242b7310e86d4687b08d680fcc824d99/roles/e0ab41d8ae22474688b5361bcc75e0f0` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:31.247341 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffb29a6-ddd8-4576-b3e0-ab7d50152e04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a5145bbe4f7423d963ad555fee26090, user_id=242b7310e86d4687b08d680fcc824d99, role_id=e0ab41d8ae22474688b5361bcc75e0f0)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:23:31.250581 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffb29a6-ddd8-4576-b3e0-ab7d50152e04 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:23:31.289648 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1217/2451] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:23:31 2022] PUT /identity/v3/projects/5a5145bbe4f7423d963ad555fee26090/users/242b7310e86d4687b08d680fcc824d99/roles/e0ab41d8ae22474688b5361bcc75e0f0 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:23:31.297336 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c698ae86-5f12-4601-b1f7-3b221ba3e07c None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:31.298868 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c698ae86-5f12-4601-b1f7-3b221ba3e07c None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:31.299400 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c698ae86-5f12-4601-b1f7-3b221ba3e07c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:31.367316 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-c698ae86-5f12-4601-b1f7-3b221ba3e07c None None] MFA Rules not processed for user `242b7310e86d4687b08d680fcc824d99`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:23:31.370378 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-c698ae86-5f12-4601-b1f7-3b221ba3e07c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:23:31.453272 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51084e2d-e7da-456c-b5d6-4575188ce29d None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:31.453570 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51084e2d-e7da-456c-b5d6-4575188ce29d None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:31.454744 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51084e2d-e7da-456c-b5d6-4575188ce29d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:31.486819 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-51084e2d-e7da-456c-b5d6-4575188ce29d None None] MFA Rules not processed for user `7f661aa1f00e4c7e916b2498505475dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:23:31.489895 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-51084e2d-e7da-456c-b5d6-4575188ce29d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:23:31.497838 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1235/2452] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:23:31 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:23:31.534891 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c698ae86-5f12-4601-b1f7-3b221ba3e07c None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:23:31.537246 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-c698ae86-5f12-4601-b1f7-3b221ba3e07c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:23:31.610047 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1218/2453] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:23:31 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:23:31.644371 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c698ae86-5f12-4601-b1f7-3b221ba3e07c None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:23:31.651693 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15dd1a2-d1d0-468a-a7f5-9acaeabdb745 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df1bca993bc142908cc7cf85eabd2b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:23:31.653100 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15dd1a2-d1d0-468a-a7f5-9acaeabdb745 service cinder] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:31.653663 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15dd1a2-d1d0-468a-a7f5-9acaeabdb745 service cinder] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:31.654285 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15dd1a2-d1d0-468a-a7f5-9acaeabdb745 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:31.674512 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15dd1a2-d1d0-468a-a7f5-9acaeabdb745 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:23:31.676878 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15dd1a2-d1d0-468a-a7f5-9acaeabdb745 service cinder] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:23:31.692194 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1236/2454] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:23:31 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:23:31.770178 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51084e2d-e7da-456c-b5d6-4575188ce29d None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:23:31.772132 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-51084e2d-e7da-456c-b5d6-4575188ce29d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:23:31.793828 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51084e2d-e7da-456c-b5d6-4575188ce29d None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:23:31.798925 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68b47a5-1021-4623-b4be-9c038a9fd4fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df1bca993bc142908cc7cf85eabd2b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:23:31.799625 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68b47a5-1021-4623-b4be-9c038a9fd4fc service cinder] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:31.799982 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68b47a5-1021-4623-b4be-9c038a9fd4fc service cinder] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:31.800321 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68b47a5-1021-4623-b4be-9c038a9fd4fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:31.914207 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68b47a5-1021-4623-b4be-9c038a9fd4fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:23:31.916674 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68b47a5-1021-4623-b4be-9c038a9fd4fc service cinder] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:23:31.930529 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1219/2455] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:23:31 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:23:36.800298 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15dd1a2-d1d0-468a-a7f5-9acaeabdb745 service cinder] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:23:36.933382 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:23:36.989561 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15dd1a2-d1d0-468a-a7f5-9acaeabdb745 service cinder] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:23:36.995859 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbfb6bf-cb88-4891-8d4d-ff042a2be43d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:23:36.996722 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbfb6bf-cb88-4891-8d4d-ff042a2be43d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:36.997135 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbfb6bf-cb88-4891-8d4d-ff042a2be43d admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:36.997522 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbfb6bf-cb88-4891-8d4d-ff042a2be43d admin admin] PATH_INFO: `/v3/users/9f89c2b4302047479f3d2abd35b53aac` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:36.999684 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbfb6bf-cb88-4891-8d4d-ff042a2be43d admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f89c2b4302047479f3d2abd35b53aac)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:23:37.003781 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbfb6bf-cb88-4891-8d4d-ff042a2be43d admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:23:37.150167 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-cfbfb6bf-cb88-4891-8d4d-ff042a2be43d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f89c2b4302047479f3d2abd35b53aac'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:23:37.151212 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-cfbfb6bf-cb88-4891-8d4d-ff042a2be43d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f89c2b4302047479f3d2abd35b53aac'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:23:37.178632 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-cfbfb6bf-cb88-4891-8d4d-ff042a2be43d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f89c2b4302047479f3d2abd35b53aac'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:23:37.180202 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-cfbfb6bf-cb88-4891-8d4d-ff042a2be43d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f89c2b4302047479f3d2abd35b53aac'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:23:37.198640 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-cfbfb6bf-cb88-4891-8d4d-ff042a2be43d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f89c2b4302047479f3d2abd35b53aac'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:23:37.230407 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1237/2456] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:23:36 2022] DELETE /identity/v3/users/9f89c2b4302047479f3d2abd35b53aac => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:23:37.236510 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773d2bf8-ea51-40bb-a8c0-a114146ab2d0 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:37.237793 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773d2bf8-ea51-40bb-a8c0-a114146ab2d0 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:37.238217 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773d2bf8-ea51-40bb-a8c0-a114146ab2d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:37.271666 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-773d2bf8-ea51-40bb-a8c0-a114146ab2d0 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:23:37.275447 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-773d2bf8-ea51-40bb-a8c0-a114146ab2d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:23:37.350706 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:23:37.410331 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1220/2457] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:23:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:23:37.422682 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbfb6bf-cb88-4891-8d4d-ff042a2be43d admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:23:37.425162 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-cfbfb6bf-cb88-4891-8d4d-ff042a2be43d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:23:37.523682 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbfb6bf-cb88-4891-8d4d-ff042a2be43d admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:23:37.530676 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87f7f38-c7b3-4331-8c34-46cf6b8230a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:23:37.531464 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87f7f38-c7b3-4331-8c34-46cf6b8230a7 service neutron] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:37.531844 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87f7f38-c7b3-4331-8c34-46cf6b8230a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:37.532325 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87f7f38-c7b3-4331-8c34-46cf6b8230a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:37.548754 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87f7f38-c7b3-4331-8c34-46cf6b8230a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:23:37.551828 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87f7f38-c7b3-4331-8c34-46cf6b8230a7 service neutron] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:23:37.572737 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1238/2458] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:23:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:23:38.122822 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773d2bf8-ea51-40bb-a8c0-a114146ab2d0 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:23:38.125167 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-773d2bf8-ea51-40bb-a8c0-a114146ab2d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:23:38.151772 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773d2bf8-ea51-40bb-a8c0-a114146ab2d0 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:23:38.161272 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4184a88b-c576-4332-91a0-82f3c3a57f84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:23:38.162741 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4184a88b-c576-4332-91a0-82f3c3a57f84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:38.163232 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4184a88b-c576-4332-91a0-82f3c3a57f84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:38.163646 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4184a88b-c576-4332-91a0-82f3c3a57f84 admin admin] PATH_INFO: `/v3/projects/019ed76a1237479fb4a69e5dec27cbde` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:38.165696 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4184a88b-c576-4332-91a0-82f3c3a57f84 admin admin] RBAC: Authorizing `identity:delete_project(project_id=019ed76a1237479fb4a69e5dec27cbde)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:23:38.167760 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4184a88b-c576-4332-91a0-82f3c3a57f84 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:23:38.236571 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-4184a88b-c576-4332-91a0-82f3c3a57f84 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '019ed76a1237479fb4a69e5dec27cbde', 'request_id': 'req-4184a88b-c576-4332-91a0-82f3c3a57f84'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:23:38.246200 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-4184a88b-c576-4332-91a0-82f3c3a57f84 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '019ed76a1237479fb4a69e5dec27cbde', 'request_id': 'req-4184a88b-c576-4332-91a0-82f3c3a57f84'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:23:38.256347 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-4184a88b-c576-4332-91a0-82f3c3a57f84 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '019ed76a1237479fb4a69e5dec27cbde', 'request_id': 'req-4184a88b-c576-4332-91a0-82f3c3a57f84'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:23:38.272890 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-4184a88b-c576-4332-91a0-82f3c3a57f84 admin admin] The token cache is being invalidate because project 019ed76a1237479fb4a69e5dec27cbde was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81771) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:23:38.273357 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-4184a88b-c576-4332-91a0-82f3c3a57f84 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:23:38.274130 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-4184a88b-c576-4332-91a0-82f3c3a57f84 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:23:38.275911 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1221/2459] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:23:38 2022] DELETE /identity/v3/projects/019ed76a1237479fb4a69e5dec27cbde => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:23:38.280713 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87f7f38-c7b3-4331-8c34-46cf6b8230a7 service neutron] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:23:38.292451 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87f7f38-c7b3-4331-8c34-46cf6b8230a7 service neutron] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:23:38.388230 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:23:38.401215 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0d9f26-6261-4300-9c40-1f54dcdf6f6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:23:38.402687 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0d9f26-6261-4300-9c40-1f54dcdf6f6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:38.403851 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0d9f26-6261-4300-9c40-1f54dcdf6f6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:38.404569 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0d9f26-6261-4300-9c40-1f54dcdf6f6f admin admin] PATH_INFO: `/v3/users/242b7310e86d4687b08d680fcc824d99` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:38.407666 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0d9f26-6261-4300-9c40-1f54dcdf6f6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=242b7310e86d4687b08d680fcc824d99)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:23:38.415791 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0d9f26-6261-4300-9c40-1f54dcdf6f6f admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:23:38.504697 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-ff0d9f26-6261-4300-9c40-1f54dcdf6f6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '242b7310e86d4687b08d680fcc824d99'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:23:38.506091 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-ff0d9f26-6261-4300-9c40-1f54dcdf6f6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '242b7310e86d4687b08d680fcc824d99'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:23:38.535238 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-ff0d9f26-6261-4300-9c40-1f54dcdf6f6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '242b7310e86d4687b08d680fcc824d99'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:23:38.536208 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-ff0d9f26-6261-4300-9c40-1f54dcdf6f6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '242b7310e86d4687b08d680fcc824d99'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:23:38.563307 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-ff0d9f26-6261-4300-9c40-1f54dcdf6f6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '242b7310e86d4687b08d680fcc824d99'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:23:38.593969 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1239/2460] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:23:38 2022] DELETE /identity/v3/users/242b7310e86d4687b08d680fcc824d99 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:23:39.045993 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4184a88b-c576-4332-91a0-82f3c3a57f84 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:23:39.049354 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-4184a88b-c576-4332-91a0-82f3c3a57f84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:23:39.075466 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4184a88b-c576-4332-91a0-82f3c3a57f84 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:23:39.185128 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef34b55-0bf8-4d41-b49d-f61b6cfaa2bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:23:39.185899 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef34b55-0bf8-4d41-b49d-f61b6cfaa2bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:39.186191 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef34b55-0bf8-4d41-b49d-f61b6cfaa2bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:39.186475 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef34b55-0bf8-4d41-b49d-f61b6cfaa2bb admin admin] PATH_INFO: `/v3/projects/5a5145bbe4f7423d963ad555fee26090` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:39.187821 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef34b55-0bf8-4d41-b49d-f61b6cfaa2bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a5145bbe4f7423d963ad555fee26090)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:23:39.189937 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef34b55-0bf8-4d41-b49d-f61b6cfaa2bb admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:23:39.264409 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-aef34b55-0bf8-4d41-b49d-f61b6cfaa2bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a5145bbe4f7423d963ad555fee26090', 'request_id': 'req-aef34b55-0bf8-4d41-b49d-f61b6cfaa2bb'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:23:39.272465 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-aef34b55-0bf8-4d41-b49d-f61b6cfaa2bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a5145bbe4f7423d963ad555fee26090', 'request_id': 'req-aef34b55-0bf8-4d41-b49d-f61b6cfaa2bb'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:23:39.278583 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:23:39.287111 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-aef34b55-0bf8-4d41-b49d-f61b6cfaa2bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a5145bbe4f7423d963ad555fee26090', 'request_id': 'req-aef34b55-0bf8-4d41-b49d-f61b6cfaa2bb'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:23:39.310313 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-aef34b55-0bf8-4d41-b49d-f61b6cfaa2bb admin admin] The token cache is being invalidate because project 5a5145bbe4f7423d963ad555fee26090 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81771) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:23:39.310807 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-aef34b55-0bf8-4d41-b49d-f61b6cfaa2bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:23:39.311993 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-aef34b55-0bf8-4d41-b49d-f61b6cfaa2bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:23:39.314385 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1222/2461] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:23:39 2022] DELETE /identity/v3/projects/5a5145bbe4f7423d963ad555fee26090 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 20:23:39.320823 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74cc510-7dc5-4c1b-832d-234255c2ba18 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:39.321807 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74cc510-7dc5-4c1b-832d-234255c2ba18 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:39.322195 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74cc510-7dc5-4c1b-832d-234255c2ba18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:39.351163 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-f74cc510-7dc5-4c1b-832d-234255c2ba18 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:23:39.352762 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-f74cc510-7dc5-4c1b-832d-234255c2ba18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:23:39.460098 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1240/2462] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:23:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:23:39.475914 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332d9e74-4d03-426f-8471-9172099919e5 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:39.476971 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332d9e74-4d03-426f-8471-9172099919e5 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:39.477439 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332d9e74-4d03-426f-8471-9172099919e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:39.514276 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-332d9e74-4d03-426f-8471-9172099919e5 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:23:39.516442 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-332d9e74-4d03-426f-8471-9172099919e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:23:39.531694 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1223/2463] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:23:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:23:39.539070 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74cc510-7dc5-4c1b-832d-234255c2ba18 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:23:39.558860 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74cc510-7dc5-4c1b-832d-234255c2ba18 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:23:39.569131 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7b62f9-df19-4899-bb33-417a8c28578f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:23:39.569677 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7b62f9-df19-4899-bb33-417a8c28578f admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:39.571062 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7b62f9-df19-4899-bb33-417a8c28578f admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:39.571062 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7b62f9-df19-4899-bb33-417a8c28578f admin admin] PATH_INFO: `/v3/domains` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:39.571798 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7b62f9-df19-4899-bb33-417a8c28578f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:23:39.572320 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7b62f9-df19-4899-bb33-417a8c28578f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:23:39.575412 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7b62f9-df19-4899-bb33-417a8c28578f admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:23:39.606919 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1241/2464] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 6 20:23:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:23:39.613644 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332d9e74-4d03-426f-8471-9172099919e5 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:23:39.629640 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332d9e74-4d03-426f-8471-9172099919e5 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:23:39.641965 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05072de-6293-414c-a4ab-1403a66bcc12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:23:39.642699 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05072de-6293-414c-a4ab-1403a66bcc12 admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:39.643052 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05072de-6293-414c-a4ab-1403a66bcc12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:39.643384 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05072de-6293-414c-a4ab-1403a66bcc12 admin admin] PATH_INFO: `/v3/projects` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:39.644075 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05072de-6293-414c-a4ab-1403a66bcc12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:23:39.647216 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05072de-6293-414c-a4ab-1403a66bcc12 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:23:39.702509 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1224/2465] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:23:39 2022] POST /identity/v3/projects => generated 381 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 20:23:39.709743 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7b62f9-df19-4899-bb33-417a8c28578f admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:23:39.736292 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7b62f9-df19-4899-bb33-417a8c28578f admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:23:39.860824 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d54fbf-0ee8-47b0-83d9-3cada13132fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:23:39.861552 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d54fbf-0ee8-47b0-83d9-3cada13132fe admin admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:39.861870 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d54fbf-0ee8-47b0-83d9-3cada13132fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:39.862200 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d54fbf-0ee8-47b0-83d9-3cada13132fe admin admin] PATH_INFO: `/v3/users` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:39.862873 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d54fbf-0ee8-47b0-83d9-3cada13132fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:23:39.866106 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d54fbf-0ee8-47b0-83d9-3cada13132fe admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:23:39.880959 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:23:39.903797 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1242/2466] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:23:39 2022] POST /identity/v3/users => generated 291 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:23:39.909952 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05072de-6293-414c-a4ab-1403a66bcc12 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:23:39.936179 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05072de-6293-414c-a4ab-1403a66bcc12 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:23:39.947782 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eda900a-ecc3-4f09-a755-815312d93d83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:23:39.949244 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eda900a-ecc3-4f09-a755-815312d93d83 admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:39.949756 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eda900a-ecc3-4f09-a755-815312d93d83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:39.950114 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eda900a-ecc3-4f09-a755-815312d93d83 admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:39.951281 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eda900a-ecc3-4f09-a755-815312d93d83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:23:39.954268 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eda900a-ecc3-4f09-a755-815312d93d83 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:23:39.973890 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1225/2467] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:23:39 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:23:39.976333 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d54fbf-0ee8-47b0-83d9-3cada13132fe admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:23:39.991009 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d54fbf-0ee8-47b0-83d9-3cada13132fe admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:23:39.997737 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975179ed-a35b-4f7e-8e2b-2517cdcadc5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:23:39.998702 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975179ed-a35b-4f7e-8e2b-2517cdcadc5e admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:39.999366 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975179ed-a35b-4f7e-8e2b-2517cdcadc5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:39.999823 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975179ed-a35b-4f7e-8e2b-2517cdcadc5e admin admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:40.000732 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975179ed-a35b-4f7e-8e2b-2517cdcadc5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:23:40.002867 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975179ed-a35b-4f7e-8e2b-2517cdcadc5e admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:23:40.022608 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1243/2468] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:23:39 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:23:40.030370 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eda900a-ecc3-4f09-a755-815312d93d83 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:23:40.054731 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eda900a-ecc3-4f09-a755-815312d93d83 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:23:40.065100 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a336da4d-6ee1-4380-aa6b-f81bfe9608c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:23:40.066705 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a336da4d-6ee1-4380-aa6b-f81bfe9608c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:40.067339 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a336da4d-6ee1-4380-aa6b-f81bfe9608c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:40.068474 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a336da4d-6ee1-4380-aa6b-f81bfe9608c2 admin admin] PATH_INFO: `/v3/projects/cf597ba7c5fd439da6d3947f87b0ad30/users/a10dbfa947b649f8b14f86eb453c16eb/roles/4eea39e24b06431aa357053fe7907a3d` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:40.104481 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a336da4d-6ee1-4380-aa6b-f81bfe9608c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf597ba7c5fd439da6d3947f87b0ad30, user_id=a10dbfa947b649f8b14f86eb453c16eb, role_id=4eea39e24b06431aa357053fe7907a3d)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:23:40.110810 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a336da4d-6ee1-4380-aa6b-f81bfe9608c2 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:23:40.154194 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1226/2469] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:23:40 2022] PUT /identity/v3/projects/cf597ba7c5fd439da6d3947f87b0ad30/users/a10dbfa947b649f8b14f86eb453c16eb/roles/4eea39e24b06431aa357053fe7907a3d => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:23:40.168785 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3496db1-22db-45ca-b79e-9c9529e86f1b None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:40.169236 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3496db1-22db-45ca-b79e-9c9529e86f1b None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:40.169780 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3496db1-22db-45ca-b79e-9c9529e86f1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:40.239481 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-c3496db1-22db-45ca-b79e-9c9529e86f1b None None] MFA Rules not processed for user `a10dbfa947b649f8b14f86eb453c16eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:23:40.239481 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-c3496db1-22db-45ca-b79e-9c9529e86f1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:23:40.401148 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1244/2470] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:23:40 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:23:40.409207 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a336da4d-6ee1-4380-aa6b-f81bfe9608c2 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:23:40.543956 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a336da4d-6ee1-4380-aa6b-f81bfe9608c2 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:23:40.549524 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4736566-06a5-4172-abee-08fdd5f7df91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:23:40.550386 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4736566-06a5-4172-abee-08fdd5f7df91 admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:40.550853 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4736566-06a5-4172-abee-08fdd5f7df91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:40.551267 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4736566-06a5-4172-abee-08fdd5f7df91 admin admin] PATH_INFO: `/v3/projects` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:40.551997 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4736566-06a5-4172-abee-08fdd5f7df91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:23:40.554650 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4736566-06a5-4172-abee-08fdd5f7df91 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:23:40.613398 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1227/2471] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:23:40 2022] POST /identity/v3/projects => generated 381 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:23:40.619126 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3496db1-22db-45ca-b79e-9c9529e86f1b None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:23:40.637187 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3496db1-22db-45ca-b79e-9c9529e86f1b None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:23:40.732498 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1436b17-fb12-4cbf-b601-4f8b8fe8857f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:23:40.733755 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1436b17-fb12-4cbf-b601-4f8b8fe8857f admin admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:40.733755 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1436b17-fb12-4cbf-b601-4f8b8fe8857f admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:40.734194 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1436b17-fb12-4cbf-b601-4f8b8fe8857f admin admin] PATH_INFO: `/v3/users` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:40.735392 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1436b17-fb12-4cbf-b601-4f8b8fe8857f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:23:40.738143 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1436b17-fb12-4cbf-b601-4f8b8fe8857f admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:23:40.781672 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1245/2472] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:23:40 2022] POST /identity/v3/users => generated 297 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:23:40.785084 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4736566-06a5-4172-abee-08fdd5f7df91 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:23:40.805198 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4736566-06a5-4172-abee-08fdd5f7df91 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:23:40.812070 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0e334b-6943-4ced-b23b-4f18591f6e30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:23:40.812070 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0e334b-6943-4ced-b23b-4f18591f6e30 admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:40.812070 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0e334b-6943-4ced-b23b-4f18591f6e30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:40.812544 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0e334b-6943-4ced-b23b-4f18591f6e30 admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:40.813295 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0e334b-6943-4ced-b23b-4f18591f6e30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:23:40.816161 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0e334b-6943-4ced-b23b-4f18591f6e30 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:23:40.833372 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1228/2473] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:23:40 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:23:40.839708 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1436b17-fb12-4cbf-b601-4f8b8fe8857f admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:23:40.873884 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1436b17-fb12-4cbf-b601-4f8b8fe8857f admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:23:40.882756 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a3996f-28fb-47da-9f32-b37650c718d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:23:40.883736 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a3996f-28fb-47da-9f32-b37650c718d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:40.884122 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a3996f-28fb-47da-9f32-b37650c718d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:40.884520 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a3996f-28fb-47da-9f32-b37650c718d4 admin admin] PATH_INFO: `/v3/projects/1c5321f906664f82a01fcd958d6af69a/users/abd4cb1e60214b0ba46c48c02f11dc06/roles/e0ab41d8ae22474688b5361bcc75e0f0` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:40.942580 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a3996f-28fb-47da-9f32-b37650c718d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c5321f906664f82a01fcd958d6af69a, user_id=abd4cb1e60214b0ba46c48c02f11dc06, role_id=e0ab41d8ae22474688b5361bcc75e0f0)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:23:40.944816 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a3996f-28fb-47da-9f32-b37650c718d4 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:23:40.979005 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1246/2474] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:23:40 2022] PUT /identity/v3/projects/1c5321f906664f82a01fcd958d6af69a/users/abd4cb1e60214b0ba46c48c02f11dc06/roles/e0ab41d8ae22474688b5361bcc75e0f0 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:23:40.986306 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0e334b-6943-4ced-b23b-4f18591f6e30 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:23:41.022178 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0e334b-6943-4ced-b23b-4f18591f6e30 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:23:41.214598 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:23:41.222979 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709924f3-c524-450d-b0d8-74c99e59978c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:23:41.223675 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709924f3-c524-450d-b0d8-74c99e59978c admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:41.224405 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709924f3-c524-450d-b0d8-74c99e59978c admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:41.225118 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709924f3-c524-450d-b0d8-74c99e59978c admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:41.225872 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709924f3-c524-450d-b0d8-74c99e59978c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:23:41.229032 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709924f3-c524-450d-b0d8-74c99e59978c admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:23:41.247705 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1229/2475] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:23:40 2022] GET /identity/v3/roles => generated 1420 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:23:41.252408 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a3996f-28fb-47da-9f32-b37650c718d4 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:23:41.267362 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a3996f-28fb-47da-9f32-b37650c718d4 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:23:41.273607 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82bb42f-777b-4aaf-9b75-5ffb4a065c84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:23:41.274509 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82bb42f-777b-4aaf-9b75-5ffb4a065c84 admin admin] REQUEST_METHOD: `PUT` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:41.274953 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82bb42f-777b-4aaf-9b75-5ffb4a065c84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:41.275301 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82bb42f-777b-4aaf-9b75-5ffb4a065c84 admin admin] PATH_INFO: `/v3/projects/1c5321f906664f82a01fcd958d6af69a/users/abd4cb1e60214b0ba46c48c02f11dc06/roles/e0ab41d8ae22474688b5361bcc75e0f0` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:41.278292 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82bb42f-777b-4aaf-9b75-5ffb4a065c84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c5321f906664f82a01fcd958d6af69a, user_id=abd4cb1e60214b0ba46c48c02f11dc06, role_id=e0ab41d8ae22474688b5361bcc75e0f0)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:23:41.282421 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82bb42f-777b-4aaf-9b75-5ffb4a065c84 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:23:41.307511 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1247/2476] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:23:41 2022] PUT /identity/v3/projects/1c5321f906664f82a01fcd958d6af69a/users/abd4cb1e60214b0ba46c48c02f11dc06/roles/e0ab41d8ae22474688b5361bcc75e0f0 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 20:23:41.313930 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd9816f-957b-4b9e-9f57-d453a7ed9815 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:41.314479 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd9816f-957b-4b9e-9f57-d453a7ed9815 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:41.315104 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd9816f-957b-4b9e-9f57-d453a7ed9815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:41.371453 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-ddd9816f-957b-4b9e-9f57-d453a7ed9815 None None] MFA Rules not processed for user `abd4cb1e60214b0ba46c48c02f11dc06`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:23:41.374658 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-ddd9816f-957b-4b9e-9f57-d453a7ed9815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:23:41.549970 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1230/2477] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:23:41 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:23:41.556746 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82bb42f-777b-4aaf-9b75-5ffb4a065c84 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:23:41.559180 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-a82bb42f-777b-4aaf-9b75-5ffb4a065c84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:23:41.595473 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:23:41.663115 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82bb42f-777b-4aaf-9b75-5ffb4a065c84 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:23:41.667488 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5876092-09ab-4bdc-a5b5-419cc3b6fde8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df1bca993bc142908cc7cf85eabd2b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:23:41.668164 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5876092-09ab-4bdc-a5b5-419cc3b6fde8 service cinder] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:41.668432 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5876092-09ab-4bdc-a5b5-419cc3b6fde8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:41.668769 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5876092-09ab-4bdc-a5b5-419cc3b6fde8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:41.680970 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5876092-09ab-4bdc-a5b5-419cc3b6fde8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:23:41.682937 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5876092-09ab-4bdc-a5b5-419cc3b6fde8 service cinder] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:23:41.696519 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1248/2478] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:23:41 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:23:42.356451 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd9816f-957b-4b9e-9f57-d453a7ed9815 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:23:42.371019 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd9816f-957b-4b9e-9f57-d453a7ed9815 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:23:42.376831 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30897ee0-cf34-448e-b895-c6b6b4d750b3 tempest-VolumeQuotaClassesTest-1800606458 tempest-VolumeQuotaClassesTest-1800606458-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abd4cb1e60214b0ba46c48c02f11dc06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c5321f906664f82a01fcd958d6af69a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:23:42.377851 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30897ee0-cf34-448e-b895-c6b6b4d750b3 tempest-VolumeQuotaClassesTest-1800606458 tempest-VolumeQuotaClassesTest-1800606458-project-admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:42.380850 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30897ee0-cf34-448e-b895-c6b6b4d750b3 tempest-VolumeQuotaClassesTest-1800606458 tempest-VolumeQuotaClassesTest-1800606458-project-admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:42.380850 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30897ee0-cf34-448e-b895-c6b6b4d750b3 tempest-VolumeQuotaClassesTest-1800606458 tempest-VolumeQuotaClassesTest-1800606458-project-admin] PATH_INFO: `/v3/domains` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:42.380850 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30897ee0-cf34-448e-b895-c6b6b4d750b3 tempest-VolumeQuotaClassesTest-1800606458 tempest-VolumeQuotaClassesTest-1800606458-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:23:42.380850 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30897ee0-cf34-448e-b895-c6b6b4d750b3 tempest-VolumeQuotaClassesTest-1800606458 tempest-VolumeQuotaClassesTest-1800606458-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:23:42.384833 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30897ee0-cf34-448e-b895-c6b6b4d750b3 tempest-VolumeQuotaClassesTest-1800606458 tempest-VolumeQuotaClassesTest-1800606458-project-admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:23:42.402463 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1231/2479] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 6 20:23:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:23:42.404643 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5876092-09ab-4bdc-a5b5-419cc3b6fde8 service cinder] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:23:42.420028 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5876092-09ab-4bdc-a5b5-419cc3b6fde8 service cinder] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:23:42.435179 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c16689-7f7a-4081-9337-02677faf9cc3 tempest-VolumeQuotaClassesTest-1800606458 tempest-VolumeQuotaClassesTest-1800606458-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abd4cb1e60214b0ba46c48c02f11dc06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c5321f906664f82a01fcd958d6af69a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:23:42.436193 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c16689-7f7a-4081-9337-02677faf9cc3 tempest-VolumeQuotaClassesTest-1800606458 tempest-VolumeQuotaClassesTest-1800606458-project-admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:42.437497 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c16689-7f7a-4081-9337-02677faf9cc3 tempest-VolumeQuotaClassesTest-1800606458 tempest-VolumeQuotaClassesTest-1800606458-project-admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:42.438301 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c16689-7f7a-4081-9337-02677faf9cc3 tempest-VolumeQuotaClassesTest-1800606458 tempest-VolumeQuotaClassesTest-1800606458-project-admin] PATH_INFO: `/v3/projects` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:42.439397 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c16689-7f7a-4081-9337-02677faf9cc3 tempest-VolumeQuotaClassesTest-1800606458 tempest-VolumeQuotaClassesTest-1800606458-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:23:42.441720 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c16689-7f7a-4081-9337-02677faf9cc3 tempest-VolumeQuotaClassesTest-1800606458 tempest-VolumeQuotaClassesTest-1800606458-project-admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:23:42.503451 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1249/2480] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:23:42 2022] POST /identity/v3/projects => generated 353 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:23:42.510520 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30897ee0-cf34-448e-b895-c6b6b4d750b3 tempest-VolumeQuotaClassesTest-1800606458 tempest-VolumeQuotaClassesTest-1800606458-project-admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:23:42.529391 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30897ee0-cf34-448e-b895-c6b6b4d750b3 tempest-VolumeQuotaClassesTest-1800606458 tempest-VolumeQuotaClassesTest-1800606458-project-admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:23:42.723424 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3810ed60-a7ce-4484-94a6-284f4adb0497 tempest-VolumeQuotaClassesTest-1800606458 tempest-VolumeQuotaClassesTest-1800606458-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abd4cb1e60214b0ba46c48c02f11dc06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c5321f906664f82a01fcd958d6af69a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:23:42.727153 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3810ed60-a7ce-4484-94a6-284f4adb0497 tempest-VolumeQuotaClassesTest-1800606458 tempest-VolumeQuotaClassesTest-1800606458-project-admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:42.727640 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3810ed60-a7ce-4484-94a6-284f4adb0497 tempest-VolumeQuotaClassesTest-1800606458 tempest-VolumeQuotaClassesTest-1800606458-project-admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:42.728156 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3810ed60-a7ce-4484-94a6-284f4adb0497 tempest-VolumeQuotaClassesTest-1800606458 tempest-VolumeQuotaClassesTest-1800606458-project-admin] PATH_INFO: `/v3/domains` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:42.729351 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3810ed60-a7ce-4484-94a6-284f4adb0497 tempest-VolumeQuotaClassesTest-1800606458 tempest-VolumeQuotaClassesTest-1800606458-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:23:42.729900 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3810ed60-a7ce-4484-94a6-284f4adb0497 tempest-VolumeQuotaClassesTest-1800606458 tempest-VolumeQuotaClassesTest-1800606458-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:23:42.734789 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3810ed60-a7ce-4484-94a6-284f4adb0497 tempest-VolumeQuotaClassesTest-1800606458 tempest-VolumeQuotaClassesTest-1800606458-project-admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:23:42.741227 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdad195b-b344-4659-9342-7f8c838f87c0 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:42.741919 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdad195b-b344-4659-9342-7f8c838f87c0 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:42.742146 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdad195b-b344-4659-9342-7f8c838f87c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:42.821001 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-bdad195b-b344-4659-9342-7f8c838f87c0 None None] MFA Rules not processed for user `1a52117df70447dba9bbee35d083e44f`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:23:42.828423 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-bdad195b-b344-4659-9342-7f8c838f87c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:23:42.835585 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1232/2481] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 6 20:23:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:23:42.928227 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3810ed60-a7ce-4484-94a6-284f4adb0497 tempest-VolumeQuotaClassesTest-1800606458 tempest-VolumeQuotaClassesTest-1800606458-project-admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:23:42.954533 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3810ed60-a7ce-4484-94a6-284f4adb0497 tempest-VolumeQuotaClassesTest-1800606458 tempest-VolumeQuotaClassesTest-1800606458-project-admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:23:42.968084 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923dff0e-a053-43f2-8dec-3345cbf50fdd tempest-VolumeQuotaClassesTest-1800606458 tempest-VolumeQuotaClassesTest-1800606458-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abd4cb1e60214b0ba46c48c02f11dc06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1c5321f906664f82a01fcd958d6af69a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:23:42.972738 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923dff0e-a053-43f2-8dec-3345cbf50fdd tempest-VolumeQuotaClassesTest-1800606458 tempest-VolumeQuotaClassesTest-1800606458-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:42.974299 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923dff0e-a053-43f2-8dec-3345cbf50fdd tempest-VolumeQuotaClassesTest-1800606458 tempest-VolumeQuotaClassesTest-1800606458-project-admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:42.976397 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923dff0e-a053-43f2-8dec-3345cbf50fdd tempest-VolumeQuotaClassesTest-1800606458 tempest-VolumeQuotaClassesTest-1800606458-project-admin] PATH_INFO: `/v3/projects/02550fe9bb68463daaeacbbb3f9c8f8e` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:42.979771 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923dff0e-a053-43f2-8dec-3345cbf50fdd tempest-VolumeQuotaClassesTest-1800606458 tempest-VolumeQuotaClassesTest-1800606458-project-admin] RBAC: Authorizing `identity:delete_project(project_id=02550fe9bb68463daaeacbbb3f9c8f8e)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:23:42.987446 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923dff0e-a053-43f2-8dec-3345cbf50fdd tempest-VolumeQuotaClassesTest-1800606458 tempest-VolumeQuotaClassesTest-1800606458-project-admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:23:43.121131 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1250/2482] 10.222.0.20 () {58 vars in 967 bytes} [Tue Sep 6 20:23:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 382 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:23:43.150421 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdad195b-b344-4659-9342-7f8c838f87c0 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:23:43.153966 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-bdad195b-b344-4659-9342-7f8c838f87c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:23:43.197488 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-923dff0e-a053-43f2-8dec-3345cbf50fdd tempest-VolumeQuotaClassesTest-1800606458 tempest-VolumeQuotaClassesTest-1800606458-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02550fe9bb68463daaeacbbb3f9c8f8e', 'request_id': 'req-923dff0e-a053-43f2-8dec-3345cbf50fdd'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:23:43.210840 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-923dff0e-a053-43f2-8dec-3345cbf50fdd tempest-VolumeQuotaClassesTest-1800606458 tempest-VolumeQuotaClassesTest-1800606458-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02550fe9bb68463daaeacbbb3f9c8f8e', 'request_id': 'req-923dff0e-a053-43f2-8dec-3345cbf50fdd'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:23:43.213772 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:23:43.239368 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-923dff0e-a053-43f2-8dec-3345cbf50fdd tempest-VolumeQuotaClassesTest-1800606458 tempest-VolumeQuotaClassesTest-1800606458-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02550fe9bb68463daaeacbbb3f9c8f8e', 'request_id': 'req-923dff0e-a053-43f2-8dec-3345cbf50fdd'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:23:43.282996 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-923dff0e-a053-43f2-8dec-3345cbf50fdd tempest-VolumeQuotaClassesTest-1800606458 tempest-VolumeQuotaClassesTest-1800606458-project-admin] The token cache is being invalidate because project 02550fe9bb68463daaeacbbb3f9c8f8e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81771) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:23:43.282996 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-923dff0e-a053-43f2-8dec-3345cbf50fdd tempest-VolumeQuotaClassesTest-1800606458 tempest-VolumeQuotaClassesTest-1800606458-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:23:43.283512 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-923dff0e-a053-43f2-8dec-3345cbf50fdd tempest-VolumeQuotaClassesTest-1800606458 tempest-VolumeQuotaClassesTest-1800606458-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:23:43.303127 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1233/2483] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:23:42 2022] DELETE /identity/v3/projects/02550fe9bb68463daaeacbbb3f9c8f8e => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:23:43.495337 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdad195b-b344-4659-9342-7f8c838f87c0 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:23:43.686859 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:23:43.758941 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be695df1-ad8a-47d9-9ff4-ce4a9c26d62f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b72516236da47e292d78de201b0552b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:23:43.761444 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be695df1-ad8a-47d9-9ff4-ce4a9c26d62f service swift] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:43.761998 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be695df1-ad8a-47d9-9ff4-ce4a9c26d62f service swift] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:43.763268 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be695df1-ad8a-47d9-9ff4-ce4a9c26d62f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:43.771465 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-be695df1-ad8a-47d9-9ff4-ce4a9c26d62f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:23:43.960416 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be695df1-ad8a-47d9-9ff4-ce4a9c26d62f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:23:43.962562 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be695df1-ad8a-47d9-9ff4-ce4a9c26d62f service swift] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:23:43.995001 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1251/2484] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Sep 6 20:23:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 846 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 20:23:44.001859 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923dff0e-a053-43f2-8dec-3345cbf50fdd tempest-VolumeQuotaClassesTest-1800606458 tempest-VolumeQuotaClassesTest-1800606458-project-admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:23:44.003653 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-923dff0e-a053-43f2-8dec-3345cbf50fdd tempest-VolumeQuotaClassesTest-1800606458 tempest-VolumeQuotaClassesTest-1800606458-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:23:44.171131 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923dff0e-a053-43f2-8dec-3345cbf50fdd tempest-VolumeQuotaClassesTest-1800606458 tempest-VolumeQuotaClassesTest-1800606458-project-admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:23:44.189782 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fce7ba6-0b03-426a-b64b-3e7d130693e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:23:44.190456 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fce7ba6-0b03-426a-b64b-3e7d130693e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:44.190735 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fce7ba6-0b03-426a-b64b-3e7d130693e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:44.191250 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fce7ba6-0b03-426a-b64b-3e7d130693e9 admin admin] PATH_INFO: `/v3/users/a10dbfa947b649f8b14f86eb453c16eb` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:44.194058 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fce7ba6-0b03-426a-b64b-3e7d130693e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a10dbfa947b649f8b14f86eb453c16eb)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:23:44.201382 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fce7ba6-0b03-426a-b64b-3e7d130693e9 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:23:44.326675 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-0fce7ba6-0b03-426a-b64b-3e7d130693e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a10dbfa947b649f8b14f86eb453c16eb'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:23:44.345528 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-0fce7ba6-0b03-426a-b64b-3e7d130693e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a10dbfa947b649f8b14f86eb453c16eb'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:23:44.346139 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-0fce7ba6-0b03-426a-b64b-3e7d130693e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a10dbfa947b649f8b14f86eb453c16eb'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:23:44.346848 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-0fce7ba6-0b03-426a-b64b-3e7d130693e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a10dbfa947b649f8b14f86eb453c16eb'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:23:44.359199 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-0fce7ba6-0b03-426a-b64b-3e7d130693e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a10dbfa947b649f8b14f86eb453c16eb'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:23:44.395477 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1234/2485] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:23:44 2022] DELETE /identity/v3/users/a10dbfa947b649f8b14f86eb453c16eb => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:23:44.401265 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a8013-7b07-453d-9219-8ba7c243b738 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:44.401720 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a8013-7b07-453d-9219-8ba7c243b738 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:44.402333 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a8013-7b07-453d-9219-8ba7c243b738 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:44.469779 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-5d7a8013-7b07-453d-9219-8ba7c243b738 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:23:44.483170 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-5d7a8013-7b07-453d-9219-8ba7c243b738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:23:44.631901 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1252/2486] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:23:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:23:44.651300 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fce7ba6-0b03-426a-b64b-3e7d130693e9 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:23:44.654596 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-0fce7ba6-0b03-426a-b64b-3e7d130693e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:23:44.740554 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fce7ba6-0b03-426a-b64b-3e7d130693e9 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:23:44.748047 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a93424-28bf-4e6e-b719-a93ff6e97082 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:23:44.748893 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a93424-28bf-4e6e-b719-a93ff6e97082 service neutron] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:44.749319 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a93424-28bf-4e6e-b719-a93ff6e97082 service neutron] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:44.749895 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a93424-28bf-4e6e-b719-a93ff6e97082 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:44.766355 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a93424-28bf-4e6e-b719-a93ff6e97082 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:23:44.768388 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a93424-28bf-4e6e-b719-a93ff6e97082 service neutron] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:23:44.794989 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1235/2487] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:23:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:23:45.439990 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c911f24-0574-43fb-84c6-264419a2b01a None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:45.441799 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c911f24-0574-43fb-84c6-264419a2b01a None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:45.443276 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c911f24-0574-43fb-84c6-264419a2b01a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:45.495029 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a93424-28bf-4e6e-b719-a93ff6e97082 service neutron] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:23:45.495633 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-1c911f24-0574-43fb-84c6-264419a2b01a None None] MFA Rules not processed for user `1a52117df70447dba9bbee35d083e44f`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:23:45.497346 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-06a93424-28bf-4e6e-b719-a93ff6e97082 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:23:45.497494 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-1c911f24-0574-43fb-84c6-264419a2b01a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:23:45.520313 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:23:45.535341 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a93424-28bf-4e6e-b719-a93ff6e97082 service neutron] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:23:45.555309 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264dda6c-4337-4459-996c-1b4e1b442370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:23:45.556133 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264dda6c-4337-4459-996c-1b4e1b442370 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:45.563137 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264dda6c-4337-4459-996c-1b4e1b442370 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:45.563855 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264dda6c-4337-4459-996c-1b4e1b442370 admin admin] PATH_INFO: `/v3/projects/cf597ba7c5fd439da6d3947f87b0ad30` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:45.565454 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264dda6c-4337-4459-996c-1b4e1b442370 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf597ba7c5fd439da6d3947f87b0ad30)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:23:45.567780 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264dda6c-4337-4459-996c-1b4e1b442370 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:23:45.665414 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1253/2488] 10.222.0.20 () {58 vars in 967 bytes} [Tue Sep 6 20:23:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:23:45.688177 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-264dda6c-4337-4459-996c-1b4e1b442370 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf597ba7c5fd439da6d3947f87b0ad30', 'request_id': 'req-264dda6c-4337-4459-996c-1b4e1b442370'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:23:45.688488 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c911f24-0574-43fb-84c6-264419a2b01a None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:23:45.689986 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-1c911f24-0574-43fb-84c6-264419a2b01a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:23:45.698841 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-264dda6c-4337-4459-996c-1b4e1b442370 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf597ba7c5fd439da6d3947f87b0ad30', 'request_id': 'req-264dda6c-4337-4459-996c-1b4e1b442370'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:23:45.718813 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-264dda6c-4337-4459-996c-1b4e1b442370 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf597ba7c5fd439da6d3947f87b0ad30', 'request_id': 'req-264dda6c-4337-4459-996c-1b4e1b442370'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:23:45.765776 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-264dda6c-4337-4459-996c-1b4e1b442370 admin admin] The token cache is being invalidate because project cf597ba7c5fd439da6d3947f87b0ad30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81771) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:23:45.766200 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-264dda6c-4337-4459-996c-1b4e1b442370 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:23:45.767308 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-264dda6c-4337-4459-996c-1b4e1b442370 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:23:45.769604 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1236/2489] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:23:45 2022] DELETE /identity/v3/projects/cf597ba7c5fd439da6d3947f87b0ad30 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 20:23:45.776115 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264dda6c-4337-4459-996c-1b4e1b442370 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:23:45.780409 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-264dda6c-4337-4459-996c-1b4e1b442370 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:23:45.833913 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:23:45.874115 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264dda6c-4337-4459-996c-1b4e1b442370 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:23:45.879724 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0cb1de-7cb9-4e58-af5d-23ce99010156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:23:45.880614 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0cb1de-7cb9-4e58-af5d-23ce99010156 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:45.880980 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0cb1de-7cb9-4e58-af5d-23ce99010156 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:45.881258 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0cb1de-7cb9-4e58-af5d-23ce99010156 admin admin] PATH_INFO: `/v3/users/abd4cb1e60214b0ba46c48c02f11dc06` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:45.882659 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0cb1de-7cb9-4e58-af5d-23ce99010156 admin admin] RBAC: Authorizing `identity:delete_user(user_id=abd4cb1e60214b0ba46c48c02f11dc06)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:23:45.884840 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0cb1de-7cb9-4e58-af5d-23ce99010156 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:23:45.898123 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c911f24-0574-43fb-84c6-264419a2b01a None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:23:45.963312 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-7f0cb1de-7cb9-4e58-af5d-23ce99010156 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abd4cb1e60214b0ba46c48c02f11dc06'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:23:45.978816 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-7f0cb1de-7cb9-4e58-af5d-23ce99010156 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abd4cb1e60214b0ba46c48c02f11dc06'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:23:45.982174 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-7f0cb1de-7cb9-4e58-af5d-23ce99010156 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abd4cb1e60214b0ba46c48c02f11dc06'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:23:45.983221 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-7f0cb1de-7cb9-4e58-af5d-23ce99010156 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abd4cb1e60214b0ba46c48c02f11dc06'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:23:45.995109 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-7f0cb1de-7cb9-4e58-af5d-23ce99010156 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abd4cb1e60214b0ba46c48c02f11dc06'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:23:46.037638 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1237/2490] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:23:45 2022] DELETE /identity/v3/users/abd4cb1e60214b0ba46c48c02f11dc06 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 20:23:46.051447 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd64db8-3b20-470b-83a9-9802a78fa27b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b72516236da47e292d78de201b0552b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:23:46.052753 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd64db8-3b20-470b-83a9-9802a78fa27b service swift] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:46.054195 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd64db8-3b20-470b-83a9-9802a78fa27b service swift] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:46.054909 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd64db8-3b20-470b-83a9-9802a78fa27b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:46.058523 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-4dd64db8-3b20-470b-83a9-9802a78fa27b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:23:46.206252 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd64db8-3b20-470b-83a9-9802a78fa27b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:23:46.209709 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd64db8-3b20-470b-83a9-9802a78fa27b service swift] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:23:46.229951 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1254/2491] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Sep 6 20:23:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 542 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 06 20:23:46.606670 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0cb1de-7cb9-4e58-af5d-23ce99010156 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:23:46.609893 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-7f0cb1de-7cb9-4e58-af5d-23ce99010156 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:23:46.628806 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:23:46.746646 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0cb1de-7cb9-4e58-af5d-23ce99010156 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:23:46.754753 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe8e64c-8562-4531-9243-a3a2496a45c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:23:46.756419 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe8e64c-8562-4531-9243-a3a2496a45c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:46.756923 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe8e64c-8562-4531-9243-a3a2496a45c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:46.757388 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe8e64c-8562-4531-9243-a3a2496a45c6 admin admin] PATH_INFO: `/v3/projects/1c5321f906664f82a01fcd958d6af69a` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:46.759622 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe8e64c-8562-4531-9243-a3a2496a45c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c5321f906664f82a01fcd958d6af69a)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:23:46.763632 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe8e64c-8562-4531-9243-a3a2496a45c6 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:23:46.863934 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-efe8e64c-8562-4531-9243-a3a2496a45c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c5321f906664f82a01fcd958d6af69a', 'request_id': 'req-efe8e64c-8562-4531-9243-a3a2496a45c6'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:23:46.873366 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-efe8e64c-8562-4531-9243-a3a2496a45c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c5321f906664f82a01fcd958d6af69a', 'request_id': 'req-efe8e64c-8562-4531-9243-a3a2496a45c6'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:23:46.884868 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-efe8e64c-8562-4531-9243-a3a2496a45c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c5321f906664f82a01fcd958d6af69a', 'request_id': 'req-efe8e64c-8562-4531-9243-a3a2496a45c6'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:23:46.907500 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-efe8e64c-8562-4531-9243-a3a2496a45c6 admin admin] The token cache is being invalidate because project 1c5321f906664f82a01fcd958d6af69a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81771) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:23:46.908038 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-efe8e64c-8562-4531-9243-a3a2496a45c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:23:46.909260 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-efe8e64c-8562-4531-9243-a3a2496a45c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:23:46.918199 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1238/2492] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:23:46 2022] DELETE /identity/v3/projects/1c5321f906664f82a01fcd958d6af69a => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:23:46.919912 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584dc361-47fc-4d91-bfb1-53a71208dbe8 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:46.920662 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584dc361-47fc-4d91-bfb1-53a71208dbe8 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:46.920662 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584dc361-47fc-4d91-bfb1-53a71208dbe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:46.943966 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-584dc361-47fc-4d91-bfb1-53a71208dbe8 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:23:46.945284 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-584dc361-47fc-4d91-bfb1-53a71208dbe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:23:47.042447 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1255/2493] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:23:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:23:47.052939 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26e27c6-df56-4115-98bb-4e9295532a03 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:47.053431 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26e27c6-df56-4115-98bb-4e9295532a03 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:47.053990 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26e27c6-df56-4115-98bb-4e9295532a03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:47.110509 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-c26e27c6-df56-4115-98bb-4e9295532a03 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:23:47.113299 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-c26e27c6-df56-4115-98bb-4e9295532a03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:23:47.131413 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1239/2494] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:23:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:23:47.132931 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584dc361-47fc-4d91-bfb1-53a71208dbe8 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:23:47.149711 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584dc361-47fc-4d91-bfb1-53a71208dbe8 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:23:47.156889 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462e514c-1e33-4726-b3ff-ee825ccbfeb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:23:47.157770 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462e514c-1e33-4726-b3ff-ee825ccbfeb5 admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:47.158153 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462e514c-1e33-4726-b3ff-ee825ccbfeb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:47.158538 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462e514c-1e33-4726-b3ff-ee825ccbfeb5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:47.159352 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462e514c-1e33-4726-b3ff-ee825ccbfeb5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:23:47.159776 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462e514c-1e33-4726-b3ff-ee825ccbfeb5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:23:47.162184 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462e514c-1e33-4726-b3ff-ee825ccbfeb5 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:23:47.179121 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1256/2495] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 6 20:23:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:23:47.185004 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26e27c6-df56-4115-98bb-4e9295532a03 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:23:47.211905 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26e27c6-df56-4115-98bb-4e9295532a03 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:23:47.223200 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b49adb-1aee-49b3-a693-c1c2906939f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:23:47.224344 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b49adb-1aee-49b3-a693-c1c2906939f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:47.224841 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b49adb-1aee-49b3-a693-c1c2906939f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:47.225328 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b49adb-1aee-49b3-a693-c1c2906939f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:47.226648 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b49adb-1aee-49b3-a693-c1c2906939f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:23:47.232104 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b49adb-1aee-49b3-a693-c1c2906939f5 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:23:47.307612 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1240/2496] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:23:47 2022] POST /identity/v3/projects => generated 393 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:23:47.313094 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462e514c-1e33-4726-b3ff-ee825ccbfeb5 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:23:47.328215 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462e514c-1e33-4726-b3ff-ee825ccbfeb5 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:23:47.341297 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=100 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:23:47.431248 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07668bd-d90f-432c-a945-53c36b6d647d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:23:47.432245 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07668bd-d90f-432c-a945-53c36b6d647d admin admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:47.432908 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07668bd-d90f-432c-a945-53c36b6d647d admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:47.433502 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07668bd-d90f-432c-a945-53c36b6d647d admin admin] PATH_INFO: `/v3/users` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:47.434783 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07668bd-d90f-432c-a945-53c36b6d647d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:23:47.436985 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07668bd-d90f-432c-a945-53c36b6d647d admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:23:47.479491 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1257/2497] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:23:47 2022] POST /identity/v3/users => generated 297 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 20:23:47.486681 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b49adb-1aee-49b3-a693-c1c2906939f5 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:23:47.500140 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b49adb-1aee-49b3-a693-c1c2906939f5 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:23:47.507387 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0a6027-f1ca-4223-90fd-386714dd92f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:23:47.508018 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0a6027-f1ca-4223-90fd-386714dd92f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:47.508721 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0a6027-f1ca-4223-90fd-386714dd92f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:47.508721 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0a6027-f1ca-4223-90fd-386714dd92f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:47.509310 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0a6027-f1ca-4223-90fd-386714dd92f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:23:47.511423 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0a6027-f1ca-4223-90fd-386714dd92f4 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:23:47.528491 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1241/2498] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:23:47 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:23:47.538054 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07668bd-d90f-432c-a945-53c36b6d647d admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:23:47.560958 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07668bd-d90f-432c-a945-53c36b6d647d admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:23:47.571098 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0344b9-9a32-4805-b3c9-b6013856f15e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:23:47.571981 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0344b9-9a32-4805-b3c9-b6013856f15e admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:47.572639 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0344b9-9a32-4805-b3c9-b6013856f15e admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:47.573130 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0344b9-9a32-4805-b3c9-b6013856f15e admin admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:47.574573 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0344b9-9a32-4805-b3c9-b6013856f15e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:23:47.577980 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0344b9-9a32-4805-b3c9-b6013856f15e admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:23:47.601239 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1258/2499] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:23:47 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 20:23:47.612871 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0a6027-f1ca-4223-90fd-386714dd92f4 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:23:47.637660 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0a6027-f1ca-4223-90fd-386714dd92f4 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:23:47.654815 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb2ac7d-6d97-41b4-9e41-293073ea5338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:23:47.657011 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb2ac7d-6d97-41b4-9e41-293073ea5338 admin admin] REQUEST_METHOD: `PUT` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:47.658034 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb2ac7d-6d97-41b4-9e41-293073ea5338 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:47.658414 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb2ac7d-6d97-41b4-9e41-293073ea5338 admin admin] PATH_INFO: `/v3/projects/1a9dc131668a40a6a50f3d2dfafffeee/users/b067cbc0dbc84813b602adca97777315/roles/4eea39e24b06431aa357053fe7907a3d` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:47.689148 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb2ac7d-6d97-41b4-9e41-293073ea5338 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a9dc131668a40a6a50f3d2dfafffeee, user_id=b067cbc0dbc84813b602adca97777315, role_id=4eea39e24b06431aa357053fe7907a3d)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:23:47.695483 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb2ac7d-6d97-41b4-9e41-293073ea5338 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:23:47.714942 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1242/2500] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:23:47 2022] PUT /identity/v3/projects/1a9dc131668a40a6a50f3d2dfafffeee/users/b067cbc0dbc84813b602adca97777315/roles/4eea39e24b06431aa357053fe7907a3d => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:23:47.727568 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc65edd2-54b2-49d8-af98-4cd45108a37c None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:47.728206 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc65edd2-54b2-49d8-af98-4cd45108a37c None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:47.728789 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc65edd2-54b2-49d8-af98-4cd45108a37c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:47.790044 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-dc65edd2-54b2-49d8-af98-4cd45108a37c None None] MFA Rules not processed for user `b067cbc0dbc84813b602adca97777315`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:23:47.792491 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-dc65edd2-54b2-49d8-af98-4cd45108a37c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:23:47.886295 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1259/2501] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:23:47 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:23:47.891010 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb2ac7d-6d97-41b4-9e41-293073ea5338 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:23:47.946810 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:23:47.994372 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb2ac7d-6d97-41b4-9e41-293073ea5338 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:23:48.000897 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bc4c93-a461-4a4c-9891-06e87a59e170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:23:48.001678 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bc4c93-a461-4a4c-9891-06e87a59e170 admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:48.002021 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bc4c93-a461-4a4c-9891-06e87a59e170 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:48.002345 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bc4c93-a461-4a4c-9891-06e87a59e170 admin admin] PATH_INFO: `/v3/projects` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:48.003146 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bc4c93-a461-4a4c-9891-06e87a59e170 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:23:48.005295 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bc4c93-a461-4a4c-9891-06e87a59e170 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:23:48.045533 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1243/2502] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:23:47 2022] POST /identity/v3/projects => generated 393 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:23:48.050060 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc65edd2-54b2-49d8-af98-4cd45108a37c None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:23:48.063822 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc65edd2-54b2-49d8-af98-4cd45108a37c None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:23:48.143517 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d2ca90-c428-4f34-a09c-8862977667d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:23:48.144274 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d2ca90-c428-4f34-a09c-8862977667d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:48.144646 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d2ca90-c428-4f34-a09c-8862977667d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:48.145026 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d2ca90-c428-4f34-a09c-8862977667d1 admin admin] PATH_INFO: `/v3/users` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:48.146436 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d2ca90-c428-4f34-a09c-8862977667d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:23:48.148574 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d2ca90-c428-4f34-a09c-8862977667d1 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:23:48.184713 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1260/2503] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:23:48 2022] POST /identity/v3/users => generated 303 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 20:23:48.189270 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bc4c93-a461-4a4c-9891-06e87a59e170 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:23:48.200696 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bc4c93-a461-4a4c-9891-06e87a59e170 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:23:48.208646 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59921ff-b33f-4e19-980f-d8300a482eee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:23:48.209862 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59921ff-b33f-4e19-980f-d8300a482eee admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:48.211521 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59921ff-b33f-4e19-980f-d8300a482eee admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:48.211521 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59921ff-b33f-4e19-980f-d8300a482eee admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:48.212102 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59921ff-b33f-4e19-980f-d8300a482eee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:23:48.215343 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59921ff-b33f-4e19-980f-d8300a482eee admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:23:48.237345 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1244/2504] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:23:48 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:23:48.242555 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d2ca90-c428-4f34-a09c-8862977667d1 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:23:48.254298 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d2ca90-c428-4f34-a09c-8862977667d1 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:23:48.261311 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ab07a6-9303-4deb-a9d1-5572cfbc8b22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:23:48.262031 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ab07a6-9303-4deb-a9d1-5572cfbc8b22 admin admin] REQUEST_METHOD: `PUT` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:48.262314 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ab07a6-9303-4deb-a9d1-5572cfbc8b22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:48.262596 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ab07a6-9303-4deb-a9d1-5572cfbc8b22 admin admin] PATH_INFO: `/v3/projects/a9477ec09f0742eab46ffe68290fe5ed/users/6e51ea1d795f467bb8e3470f457edb2a/roles/e0ab41d8ae22474688b5361bcc75e0f0` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:48.266509 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:23:48.284270 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ab07a6-9303-4deb-a9d1-5572cfbc8b22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9477ec09f0742eab46ffe68290fe5ed, user_id=6e51ea1d795f467bb8e3470f457edb2a, role_id=e0ab41d8ae22474688b5361bcc75e0f0)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:23:48.287699 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ab07a6-9303-4deb-a9d1-5572cfbc8b22 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:23:48.303694 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1261/2505] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:23:48 2022] PUT /identity/v3/projects/a9477ec09f0742eab46ffe68290fe5ed/users/6e51ea1d795f467bb8e3470f457edb2a/roles/e0ab41d8ae22474688b5361bcc75e0f0 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 20:23:48.309156 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59921ff-b33f-4e19-980f-d8300a482eee admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:23:48.330918 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59921ff-b33f-4e19-980f-d8300a482eee admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:23:48.480967 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46e2542-0652-4023-a1ee-af2b6caef775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:23:48.480967 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46e2542-0652-4023-a1ee-af2b6caef775 admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:48.480967 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46e2542-0652-4023-a1ee-af2b6caef775 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:48.481591 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46e2542-0652-4023-a1ee-af2b6caef775 admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:48.481750 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46e2542-0652-4023-a1ee-af2b6caef775 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:23:48.484101 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46e2542-0652-4023-a1ee-af2b6caef775 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:23:48.499475 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1245/2506] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:23:48 2022] GET /identity/v3/roles => generated 1420 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:23:48.508898 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ab07a6-9303-4deb-a9d1-5572cfbc8b22 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:23:48.544527 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ab07a6-9303-4deb-a9d1-5572cfbc8b22 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:23:48.565368 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca0fcaa-84bc-4e17-8544-c67712574923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:23:48.567202 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca0fcaa-84bc-4e17-8544-c67712574923 admin admin] REQUEST_METHOD: `PUT` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:48.567832 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca0fcaa-84bc-4e17-8544-c67712574923 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:48.569185 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca0fcaa-84bc-4e17-8544-c67712574923 admin admin] PATH_INFO: `/v3/projects/a9477ec09f0742eab46ffe68290fe5ed/users/6e51ea1d795f467bb8e3470f457edb2a/roles/e0ab41d8ae22474688b5361bcc75e0f0` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:48.572818 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca0fcaa-84bc-4e17-8544-c67712574923 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9477ec09f0742eab46ffe68290fe5ed, user_id=6e51ea1d795f467bb8e3470f457edb2a, role_id=e0ab41d8ae22474688b5361bcc75e0f0)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:23:48.576338 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca0fcaa-84bc-4e17-8544-c67712574923 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:23:48.601687 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1262/2507] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:23:48 2022] PUT /identity/v3/projects/a9477ec09f0742eab46ffe68290fe5ed/users/6e51ea1d795f467bb8e3470f457edb2a/roles/e0ab41d8ae22474688b5361bcc75e0f0 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:23:48.628773 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445e9ef3-75aa-4201-b311-924f90762ac8 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:48.628773 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445e9ef3-75aa-4201-b311-924f90762ac8 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:48.628773 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445e9ef3-75aa-4201-b311-924f90762ac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:48.690912 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-445e9ef3-75aa-4201-b311-924f90762ac8 None None] MFA Rules not processed for user `6e51ea1d795f467bb8e3470f457edb2a`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:23:48.694134 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-445e9ef3-75aa-4201-b311-924f90762ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:23:48.825148 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1246/2508] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:23:48 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:23:48.842825 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca0fcaa-84bc-4e17-8544-c67712574923 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:23:48.846435 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-5ca0fcaa-84bc-4e17-8544-c67712574923 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:23:48.988538 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca0fcaa-84bc-4e17-8544-c67712574923 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:23:48.995754 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57b6169-e289-4286-bcdc-21e32b2775b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df1bca993bc142908cc7cf85eabd2b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:23:48.997445 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57b6169-e289-4286-bcdc-21e32b2775b6 service cinder] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:48.998226 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57b6169-e289-4286-bcdc-21e32b2775b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:48.998786 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57b6169-e289-4286-bcdc-21e32b2775b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:49.017157 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57b6169-e289-4286-bcdc-21e32b2775b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:23:49.018874 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57b6169-e289-4286-bcdc-21e32b2775b6 service cinder] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:23:49.034162 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1263/2509] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:23:48 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:23:49.134163 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445e9ef3-75aa-4201-b311-924f90762ac8 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:23:49.136386 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-445e9ef3-75aa-4201-b311-924f90762ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:23:49.166745 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445e9ef3-75aa-4201-b311-924f90762ac8 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:23:49.171333 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7163e8be-7d24-4272-96ad-067e8647148d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df1bca993bc142908cc7cf85eabd2b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:23:49.172016 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7163e8be-7d24-4272-96ad-067e8647148d service cinder] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:49.172343 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7163e8be-7d24-4272-96ad-067e8647148d service cinder] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:49.172678 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7163e8be-7d24-4272-96ad-067e8647148d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:49.241763 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:23:49.314687 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7163e8be-7d24-4272-96ad-067e8647148d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:23:49.317783 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7163e8be-7d24-4272-96ad-067e8647148d service cinder] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:23:49.336734 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1247/2510] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:23:49 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 20:23:51.439177 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57b6169-e289-4286-bcdc-21e32b2775b6 service cinder] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:23:51.440771 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-f57b6169-e289-4286-bcdc-21e32b2775b6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:23:51.577016 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57b6169-e289-4286-bcdc-21e32b2775b6 service cinder] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:23:51.585195 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793d0725-7ca7-44a7-b49e-2466ab093fdb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:23:51.586143 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793d0725-7ca7-44a7-b49e-2466ab093fdb service neutron] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:51.586733 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793d0725-7ca7-44a7-b49e-2466ab093fdb service neutron] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:51.587439 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793d0725-7ca7-44a7-b49e-2466ab093fdb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:51.590304 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-793d0725-7ca7-44a7-b49e-2466ab093fdb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:23:51.729216 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793d0725-7ca7-44a7-b49e-2466ab093fdb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:23:51.732557 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793d0725-7ca7-44a7-b49e-2466ab093fdb service neutron] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:23:51.753919 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1264/2511] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:23:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:23:52.758035 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7163e8be-7d24-4272-96ad-067e8647148d service cinder] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:23:52.760477 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-7163e8be-7d24-4272-96ad-067e8647148d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:23:53.038667 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7163e8be-7d24-4272-96ad-067e8647148d service cinder] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:23:53.058827 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1128a31-fd4c-44a8-9dd7-bbe4cb3f023d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f57872de7bb74ea48d30e93e51830aba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:23:53.059996 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1128a31-fd4c-44a8-9dd7-bbe4cb3f023d service placement] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:53.061578 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1128a31-fd4c-44a8-9dd7-bbe4cb3f023d service placement] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:53.063109 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1128a31-fd4c-44a8-9dd7-bbe4cb3f023d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:53.068284 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-e1128a31-fd4c-44a8-9dd7-bbe4cb3f023d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:23:53.118737 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1128a31-fd4c-44a8-9dd7-bbe4cb3f023d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:23:53.120507 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1128a31-fd4c-44a8-9dd7-bbe4cb3f023d service placement] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:23:53.138328 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1248/2512] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:23:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:23:54.356406 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793d0725-7ca7-44a7-b49e-2466ab093fdb service neutron] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:23:54.372147 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793d0725-7ca7-44a7-b49e-2466ab093fdb service neutron] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:23:54.388830 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5e907b-fd73-4a83-9dc8-ee6c18f07797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:23:54.391208 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5e907b-fd73-4a83-9dc8-ee6c18f07797 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:54.392430 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5e907b-fd73-4a83-9dc8-ee6c18f07797 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:54.393025 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5e907b-fd73-4a83-9dc8-ee6c18f07797 admin admin] PATH_INFO: `/v3/users/b067cbc0dbc84813b602adca97777315` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:54.418868 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5e907b-fd73-4a83-9dc8-ee6c18f07797 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b067cbc0dbc84813b602adca97777315)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:23:54.429366 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5e907b-fd73-4a83-9dc8-ee6c18f07797 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:23:54.509989 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:23:54.586530 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-ab5e907b-fd73-4a83-9dc8-ee6c18f07797 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b067cbc0dbc84813b602adca97777315'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:23:54.626710 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-ab5e907b-fd73-4a83-9dc8-ee6c18f07797 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b067cbc0dbc84813b602adca97777315'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:23:54.660304 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-ab5e907b-fd73-4a83-9dc8-ee6c18f07797 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b067cbc0dbc84813b602adca97777315'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:23:54.668685 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-ab5e907b-fd73-4a83-9dc8-ee6c18f07797 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b067cbc0dbc84813b602adca97777315'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:23:54.699357 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-ab5e907b-fd73-4a83-9dc8-ee6c18f07797 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b067cbc0dbc84813b602adca97777315'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:23:54.744284 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1265/2513] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:23:54 2022] DELETE /identity/v3/users/b067cbc0dbc84813b602adca97777315 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 20:23:54.747168 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7fab04-dec4-49a6-9bf5-07632c2f491b None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:54.748003 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7fab04-dec4-49a6-9bf5-07632c2f491b None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:54.748162 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7fab04-dec4-49a6-9bf5-07632c2f491b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:54.816571 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-ed7fab04-dec4-49a6-9bf5-07632c2f491b None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:23:54.820503 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-ed7fab04-dec4-49a6-9bf5-07632c2f491b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:23:55.026355 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1249/2514] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:23:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:23:55.211749 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5e907b-fd73-4a83-9dc8-ee6c18f07797 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:23:55.215661 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-ab5e907b-fd73-4a83-9dc8-ee6c18f07797 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:23:55.359138 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5e907b-fd73-4a83-9dc8-ee6c18f07797 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:23:55.380754 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eed5aa8-dddd-4a01-ab6d-3f529353f748 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:23:55.381787 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eed5aa8-dddd-4a01-ab6d-3f529353f748 service neutron] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:55.382237 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eed5aa8-dddd-4a01-ab6d-3f529353f748 service neutron] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:55.382789 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eed5aa8-dddd-4a01-ab6d-3f529353f748 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:55.408007 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eed5aa8-dddd-4a01-ab6d-3f529353f748 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:23:55.412340 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eed5aa8-dddd-4a01-ab6d-3f529353f748 service neutron] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:23:55.444538 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1266/2515] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:23:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 20:23:56.186085 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7fab04-dec4-49a6-9bf5-07632c2f491b None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:23:56.187624 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-ed7fab04-dec4-49a6-9bf5-07632c2f491b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:23:56.201693 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7fab04-dec4-49a6-9bf5-07632c2f491b None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:23:56.208325 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99e5fdc-9486-4121-ad4f-4e0ea2040379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:23:56.209458 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99e5fdc-9486-4121-ad4f-4e0ea2040379 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:56.210057 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99e5fdc-9486-4121-ad4f-4e0ea2040379 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:56.210617 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99e5fdc-9486-4121-ad4f-4e0ea2040379 admin admin] PATH_INFO: `/v3/projects/1a9dc131668a40a6a50f3d2dfafffeee` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:56.212242 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99e5fdc-9486-4121-ad4f-4e0ea2040379 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a9dc131668a40a6a50f3d2dfafffeee)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:23:56.214711 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99e5fdc-9486-4121-ad4f-4e0ea2040379 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:23:56.328694 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-c99e5fdc-9486-4121-ad4f-4e0ea2040379 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a9dc131668a40a6a50f3d2dfafffeee', 'request_id': 'req-c99e5fdc-9486-4121-ad4f-4e0ea2040379'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:23:56.333382 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:23:56.348585 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-c99e5fdc-9486-4121-ad4f-4e0ea2040379 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a9dc131668a40a6a50f3d2dfafffeee', 'request_id': 'req-c99e5fdc-9486-4121-ad4f-4e0ea2040379'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:23:56.373029 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-c99e5fdc-9486-4121-ad4f-4e0ea2040379 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a9dc131668a40a6a50f3d2dfafffeee', 'request_id': 'req-c99e5fdc-9486-4121-ad4f-4e0ea2040379'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:23:56.409739 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-c99e5fdc-9486-4121-ad4f-4e0ea2040379 admin admin] The token cache is being invalidate because project 1a9dc131668a40a6a50f3d2dfafffeee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81771) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:23:56.410844 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-c99e5fdc-9486-4121-ad4f-4e0ea2040379 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:23:56.414901 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-c99e5fdc-9486-4121-ad4f-4e0ea2040379 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:23:56.418905 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1250/2516] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:23:56 2022] DELETE /identity/v3/projects/1a9dc131668a40a6a50f3d2dfafffeee => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:23:56.424520 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eed5aa8-dddd-4a01-ab6d-3f529353f748 service neutron] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:23:56.452535 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eed5aa8-dddd-4a01-ab6d-3f529353f748 service neutron] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:23:56.605882 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d50e1c4-9b63-4ccd-8b24-333ab905c7e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:23:56.606558 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d50e1c4-9b63-4ccd-8b24-333ab905c7e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:56.606846 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d50e1c4-9b63-4ccd-8b24-333ab905c7e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:56.607130 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d50e1c4-9b63-4ccd-8b24-333ab905c7e3 admin admin] PATH_INFO: `/v3/users/6e51ea1d795f467bb8e3470f457edb2a` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:56.616398 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d50e1c4-9b63-4ccd-8b24-333ab905c7e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e51ea1d795f467bb8e3470f457edb2a)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:23:56.618699 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d50e1c4-9b63-4ccd-8b24-333ab905c7e3 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:23:56.738660 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:23:56.752519 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-0d50e1c4-9b63-4ccd-8b24-333ab905c7e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e51ea1d795f467bb8e3470f457edb2a'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:23:56.756640 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-0d50e1c4-9b63-4ccd-8b24-333ab905c7e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e51ea1d795f467bb8e3470f457edb2a'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:23:56.784455 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-0d50e1c4-9b63-4ccd-8b24-333ab905c7e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e51ea1d795f467bb8e3470f457edb2a'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:23:56.785557 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-0d50e1c4-9b63-4ccd-8b24-333ab905c7e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e51ea1d795f467bb8e3470f457edb2a'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:23:56.800672 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-0d50e1c4-9b63-4ccd-8b24-333ab905c7e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e51ea1d795f467bb8e3470f457edb2a'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:23:56.825256 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1267/2517] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:23:56 2022] DELETE /identity/v3/users/6e51ea1d795f467bb8e3470f457edb2a => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:23:57.746934 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99e5fdc-9486-4121-ad4f-4e0ea2040379 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:23:57.749886 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-c99e5fdc-9486-4121-ad4f-4e0ea2040379 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:23:57.852830 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99e5fdc-9486-4121-ad4f-4e0ea2040379 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:23:57.949660 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bb9280-4f30-4dd3-ae63-1e8094501147 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:23:57.950912 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bb9280-4f30-4dd3-ae63-1e8094501147 service neutron] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:57.951465 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bb9280-4f30-4dd3-ae63-1e8094501147 service neutron] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:57.952009 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bb9280-4f30-4dd3-ae63-1e8094501147 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:57.955065 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-04bb9280-4f30-4dd3-ae63-1e8094501147 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:23:57.975568 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d50e1c4-9b63-4ccd-8b24-333ab905c7e3 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:23:57.978920 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-0d50e1c4-9b63-4ccd-8b24-333ab905c7e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:23:58.075339 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bb9280-4f30-4dd3-ae63-1e8094501147 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:23:58.077624 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bb9280-4f30-4dd3-ae63-1e8094501147 service neutron] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:23:58.094536 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1251/2518] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:23:57 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:23:58.144724 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d50e1c4-9b63-4ccd-8b24-333ab905c7e3 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:23:58.167063 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3f5e13-6e4e-46af-b538-ffe17efcb731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:23:58.168261 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3f5e13-6e4e-46af-b538-ffe17efcb731 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:58.168611 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3f5e13-6e4e-46af-b538-ffe17efcb731 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:58.169118 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3f5e13-6e4e-46af-b538-ffe17efcb731 admin admin] PATH_INFO: `/v3/projects/a9477ec09f0742eab46ffe68290fe5ed` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:58.174164 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3f5e13-6e4e-46af-b538-ffe17efcb731 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9477ec09f0742eab46ffe68290fe5ed)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:23:58.178171 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3f5e13-6e4e-46af-b538-ffe17efcb731 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:23:58.289042 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-4f3f5e13-6e4e-46af-b538-ffe17efcb731 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9477ec09f0742eab46ffe68290fe5ed', 'request_id': 'req-4f3f5e13-6e4e-46af-b538-ffe17efcb731'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:23:58.305298 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-4f3f5e13-6e4e-46af-b538-ffe17efcb731 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9477ec09f0742eab46ffe68290fe5ed', 'request_id': 'req-4f3f5e13-6e4e-46af-b538-ffe17efcb731'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:23:58.323902 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-4f3f5e13-6e4e-46af-b538-ffe17efcb731 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9477ec09f0742eab46ffe68290fe5ed', 'request_id': 'req-4f3f5e13-6e4e-46af-b538-ffe17efcb731'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:23:58.350295 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-4f3f5e13-6e4e-46af-b538-ffe17efcb731 admin admin] The token cache is being invalidate because project a9477ec09f0742eab46ffe68290fe5ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81772) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:23:58.351092 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-4f3f5e13-6e4e-46af-b538-ffe17efcb731 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:23:58.353069 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-4f3f5e13-6e4e-46af-b538-ffe17efcb731 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:23:58.357402 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1268/2519] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:23:57 2022] DELETE /identity/v3/projects/a9477ec09f0742eab46ffe68290fe5ed => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 20:23:58.365239 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657042ad-88c2-4ff1-9a43-dbc56313bcd1 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:58.374631 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657042ad-88c2-4ff1-9a43-dbc56313bcd1 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:58.376629 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657042ad-88c2-4ff1-9a43-dbc56313bcd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:58.447488 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-657042ad-88c2-4ff1-9a43-dbc56313bcd1 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:23:58.450973 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-657042ad-88c2-4ff1-9a43-dbc56313bcd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:23:58.658582 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:23:58.720094 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1252/2520] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:23:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 358 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:23:58.744812 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f954bc-d0fb-4058-acad-f6141e24a665 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:58.744812 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f954bc-d0fb-4058-acad-f6141e24a665 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:58.744812 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f954bc-d0fb-4058-acad-f6141e24a665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:58.809735 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-91f954bc-d0fb-4058-acad-f6141e24a665 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:23:58.812463 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-91f954bc-d0fb-4058-acad-f6141e24a665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:23:58.845005 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657042ad-88c2-4ff1-9a43-dbc56313bcd1 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:23:58.858453 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1269/2521] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:23:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:23:58.877649 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657042ad-88c2-4ff1-9a43-dbc56313bcd1 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:23:58.890017 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc98136c-2e0b-4c56-8340-fab2f9fa62e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:23:58.891868 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc98136c-2e0b-4c56-8340-fab2f9fa62e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:58.892351 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc98136c-2e0b-4c56-8340-fab2f9fa62e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:58.892889 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc98136c-2e0b-4c56-8340-fab2f9fa62e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:58.898463 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc98136c-2e0b-4c56-8340-fab2f9fa62e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:23:58.899109 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc98136c-2e0b-4c56-8340-fab2f9fa62e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:23:58.903601 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc98136c-2e0b-4c56-8340-fab2f9fa62e8 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:23:58.942333 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1253/2522] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 6 20:23:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:23:58.949772 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f954bc-d0fb-4058-acad-f6141e24a665 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:23:58.977501 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f954bc-d0fb-4058-acad-f6141e24a665 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:23:58.990679 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289fa655-4b7a-44e7-88f9-e775a8e07a9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:23:58.992734 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289fa655-4b7a-44e7-88f9-e775a8e07a9d admin admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:58.993272 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289fa655-4b7a-44e7-88f9-e775a8e07a9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:58.994414 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289fa655-4b7a-44e7-88f9-e775a8e07a9d admin admin] PATH_INFO: `/v3/projects` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:58.996109 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289fa655-4b7a-44e7-88f9-e775a8e07a9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:23:59.001734 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289fa655-4b7a-44e7-88f9-e775a8e07a9d admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:23:59.097605 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1270/2523] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:23:58 2022] POST /identity/v3/projects => generated 401 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:23:59.104087 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc98136c-2e0b-4c56-8340-fab2f9fa62e8 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:23:59.161525 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc98136c-2e0b-4c56-8340-fab2f9fa62e8 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:23:59.314723 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f67562-1a4d-4c8a-ade7-ff0a6ebc34c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:23:59.314723 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f67562-1a4d-4c8a-ade7-ff0a6ebc34c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:59.315257 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f67562-1a4d-4c8a-ade7-ff0a6ebc34c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:59.315257 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f67562-1a4d-4c8a-ade7-ff0a6ebc34c5 admin admin] PATH_INFO: `/v3/users` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:59.315861 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f67562-1a4d-4c8a-ade7-ff0a6ebc34c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:23:59.318323 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f67562-1a4d-4c8a-ade7-ff0a6ebc34c5 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:23:59.350128 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1254/2524] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:23:59 2022] POST /identity/v3/users => generated 301 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:23:59.355836 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289fa655-4b7a-44e7-88f9-e775a8e07a9d admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:23:59.369757 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289fa655-4b7a-44e7-88f9-e775a8e07a9d admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:23:59.376043 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5629b50a-2151-4052-995d-47d7919b7402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:23:59.376687 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5629b50a-2151-4052-995d-47d7919b7402 admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:59.376977 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5629b50a-2151-4052-995d-47d7919b7402 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:59.377249 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5629b50a-2151-4052-995d-47d7919b7402 admin admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:59.378114 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5629b50a-2151-4052-995d-47d7919b7402 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:23:59.380872 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5629b50a-2151-4052-995d-47d7919b7402 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:23:59.397838 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1271/2525] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:23:59 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:23:59.403492 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f67562-1a4d-4c8a-ade7-ff0a6ebc34c5 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:23:59.423175 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f67562-1a4d-4c8a-ade7-ff0a6ebc34c5 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:23:59.439179 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5316ffa7-80e8-41d4-91e7-203aa2ebceda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:23:59.442177 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5316ffa7-80e8-41d4-91e7-203aa2ebceda admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:59.443406 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5316ffa7-80e8-41d4-91e7-203aa2ebceda admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:59.444174 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5316ffa7-80e8-41d4-91e7-203aa2ebceda admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:59.447262 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5316ffa7-80e8-41d4-91e7-203aa2ebceda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:23:59.451773 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5316ffa7-80e8-41d4-91e7-203aa2ebceda admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:23:59.534128 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1255/2526] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:23:59 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:23:59.534399 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5629b50a-2151-4052-995d-47d7919b7402 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:23:59.564578 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5629b50a-2151-4052-995d-47d7919b7402 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:23:59.582255 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98011c41-9273-42e6-822e-b72d942c6585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:23:59.583362 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98011c41-9273-42e6-822e-b72d942c6585 admin admin] REQUEST_METHOD: `PUT` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:59.583848 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98011c41-9273-42e6-822e-b72d942c6585 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:59.584638 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98011c41-9273-42e6-822e-b72d942c6585 admin admin] PATH_INFO: `/v3/projects/8683e525130f41479467d887f438da8e/users/f00e10c87983498dbab6011a41f5e1da/roles/4eea39e24b06431aa357053fe7907a3d` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:59.593905 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:23:59.624392 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98011c41-9273-42e6-822e-b72d942c6585 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8683e525130f41479467d887f438da8e, user_id=f00e10c87983498dbab6011a41f5e1da, role_id=4eea39e24b06431aa357053fe7907a3d)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:23:59.627766 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98011c41-9273-42e6-822e-b72d942c6585 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:23:59.677674 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1272/2527] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:23:59 2022] PUT /identity/v3/projects/8683e525130f41479467d887f438da8e/users/f00e10c87983498dbab6011a41f5e1da/roles/4eea39e24b06431aa357053fe7907a3d => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:23:59.694914 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594670d5-d9d0-42a5-853f-013faaf8c7e4 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:23:59.695216 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594670d5-d9d0-42a5-853f-013faaf8c7e4 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:23:59.695423 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594670d5-d9d0-42a5-853f-013faaf8c7e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:23:59.783182 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-594670d5-d9d0-42a5-853f-013faaf8c7e4 None None] MFA Rules not processed for user `f00e10c87983498dbab6011a41f5e1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:23:59.783966 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-594670d5-d9d0-42a5-853f-013faaf8c7e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:23:59.989512 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98011c41-9273-42e6-822e-b72d942c6585 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:23:59.993504 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-98011c41-9273-42e6-822e-b72d942c6585 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:00.042962 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1256/2528] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:23:59 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 349 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:24:00.048843 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594670d5-d9d0-42a5-853f-013faaf8c7e4 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:00.171107 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:24:00.187866 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98011c41-9273-42e6-822e-b72d942c6585 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:00.207405 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cba6b50-be95-4571-b967-daa72fdfc462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:00.207909 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cba6b50-be95-4571-b967-daa72fdfc462 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:00.208113 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cba6b50-be95-4571-b967-daa72fdfc462 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:00.208394 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cba6b50-be95-4571-b967-daa72fdfc462 admin admin] PATH_INFO: `/v3/users/ad400ce73b644938aef19e2da731c4dc` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:00.211965 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cba6b50-be95-4571-b967-daa72fdfc462 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad400ce73b644938aef19e2da731c4dc)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:00.214404 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cba6b50-be95-4571-b967-daa72fdfc462 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:00.226544 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594670d5-d9d0-42a5-853f-013faaf8c7e4 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:00.232485 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30f237c-f876-4f56-b536-e87945c2eb94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:00.233376 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30f237c-f876-4f56-b536-e87945c2eb94 admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:00.233881 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30f237c-f876-4f56-b536-e87945c2eb94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:00.235077 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30f237c-f876-4f56-b536-e87945c2eb94 admin admin] PATH_INFO: `/v3/projects` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:00.236575 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30f237c-f876-4f56-b536-e87945c2eb94 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:00.240879 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30f237c-f876-4f56-b536-e87945c2eb94 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:00.311700 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1257/2529] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:24:00 2022] POST /identity/v3/projects => generated 399 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:24:00.321564 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30f237c-f876-4f56-b536-e87945c2eb94 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:00.335095 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-7cba6b50-be95-4571-b967-daa72fdfc462 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad400ce73b644938aef19e2da731c4dc'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:00.336568 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-7cba6b50-be95-4571-b967-daa72fdfc462 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad400ce73b644938aef19e2da731c4dc'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:00.394397 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-7cba6b50-be95-4571-b967-daa72fdfc462 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad400ce73b644938aef19e2da731c4dc'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:00.396748 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-7cba6b50-be95-4571-b967-daa72fdfc462 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad400ce73b644938aef19e2da731c4dc'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:00.415333 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-7cba6b50-be95-4571-b967-daa72fdfc462 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad400ce73b644938aef19e2da731c4dc'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:00.457056 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1273/2530] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:23:59 2022] DELETE /identity/v3/users/ad400ce73b644938aef19e2da731c4dc => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:24:00.461254 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cba6b50-be95-4571-b967-daa72fdfc462 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:00.464247 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-7cba6b50-be95-4571-b967-daa72fdfc462 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:00.516979 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30f237c-f876-4f56-b536-e87945c2eb94 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:00.634482 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cba6b50-be95-4571-b967-daa72fdfc462 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:00.639388 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a1bffb-a7d5-4204-88fb-3b909320e4e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:00.641755 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a1bffb-a7d5-4204-88fb-3b909320e4e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:00.642586 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a1bffb-a7d5-4204-88fb-3b909320e4e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:00.642883 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a1bffb-a7d5-4204-88fb-3b909320e4e8 admin admin] PATH_INFO: `/v3/users` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:00.646073 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a1bffb-a7d5-4204-88fb-3b909320e4e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:00.650491 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a1bffb-a7d5-4204-88fb-3b909320e4e8 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:00.651427 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19636597-3c91-4db3-8227-1a67fd2e1b82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:00.652377 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19636597-3c91-4db3-8227-1a67fd2e1b82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:00.652656 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19636597-3c91-4db3-8227-1a67fd2e1b82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:00.652934 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19636597-3c91-4db3-8227-1a67fd2e1b82 admin admin] PATH_INFO: `/v3/users/a385e6b1f23d449f8738a6f87720184d` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:00.654266 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19636597-3c91-4db3-8227-1a67fd2e1b82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a385e6b1f23d449f8738a6f87720184d)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:00.657129 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19636597-3c91-4db3-8227-1a67fd2e1b82 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:00.664436 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:24:00.696264 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1258/2531] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:24:00 2022] POST /identity/v3/users => generated 306 bytes in 384 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:24:00.702119 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a1bffb-a7d5-4204-88fb-3b909320e4e8 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:00.705995 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-56a1bffb-a7d5-4204-88fb-3b909320e4e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:00.719702 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a1bffb-a7d5-4204-88fb-3b909320e4e8 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:00.733600 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cabc1c-0c5f-42d9-b487-e23172e362bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:00.734368 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cabc1c-0c5f-42d9-b487-e23172e362bb admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:00.734654 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cabc1c-0c5f-42d9-b487-e23172e362bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:00.734944 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cabc1c-0c5f-42d9-b487-e23172e362bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:00.735626 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cabc1c-0c5f-42d9-b487-e23172e362bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:00.737270 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cabc1c-0c5f-42d9-b487-e23172e362bb admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:00.748921 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-19636597-3c91-4db3-8227-1a67fd2e1b82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a385e6b1f23d449f8738a6f87720184d'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:00.751719 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1259/2532] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:24:00 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:24:00.753912 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-19636597-3c91-4db3-8227-1a67fd2e1b82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a385e6b1f23d449f8738a6f87720184d'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:00.757062 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cabc1c-0c5f-42d9-b487-e23172e362bb admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:00.780011 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cabc1c-0c5f-42d9-b487-e23172e362bb admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:00.791719 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-19636597-3c91-4db3-8227-1a67fd2e1b82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a385e6b1f23d449f8738a6f87720184d'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:00.793219 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-19636597-3c91-4db3-8227-1a67fd2e1b82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a385e6b1f23d449f8738a6f87720184d'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:00.797584 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f35c7e-a507-4e83-8814-81f3843168dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:00.799305 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f35c7e-a507-4e83-8814-81f3843168dc admin admin] REQUEST_METHOD: `PUT` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:00.800132 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f35c7e-a507-4e83-8814-81f3843168dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:00.800800 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f35c7e-a507-4e83-8814-81f3843168dc admin admin] PATH_INFO: `/v3/projects/be3586699a5a4c6cb27ca651b7854e1f/users/8969fadfef20424bbc28101ca3d7ae06/roles/e0ab41d8ae22474688b5361bcc75e0f0` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:00.813890 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-19636597-3c91-4db3-8227-1a67fd2e1b82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a385e6b1f23d449f8738a6f87720184d'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:00.837547 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f35c7e-a507-4e83-8814-81f3843168dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=be3586699a5a4c6cb27ca651b7854e1f, user_id=8969fadfef20424bbc28101ca3d7ae06, role_id=e0ab41d8ae22474688b5361bcc75e0f0)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:00.841248 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f35c7e-a507-4e83-8814-81f3843168dc admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:00.866725 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1274/2533] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:24:00 2022] DELETE /identity/v3/users/a385e6b1f23d449f8738a6f87720184d => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:24:00.868600 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19636597-3c91-4db3-8227-1a67fd2e1b82 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:00.870997 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-19636597-3c91-4db3-8227-1a67fd2e1b82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:00.875728 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1260/2534] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:24:00 2022] PUT /identity/v3/projects/be3586699a5a4c6cb27ca651b7854e1f/users/8969fadfef20424bbc28101ca3d7ae06/roles/e0ab41d8ae22474688b5361bcc75e0f0 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 20:24:00.881021 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f35c7e-a507-4e83-8814-81f3843168dc admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:00.884673 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-94f35c7e-a507-4e83-8814-81f3843168dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:00.986501 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:24:01.053530 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19636597-3c91-4db3-8227-1a67fd2e1b82 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:01.061129 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd1d78f-ff4f-4f94-b439-443b1161a963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:01.061925 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd1d78f-ff4f-4f94-b439-443b1161a963 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:01.062314 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd1d78f-ff4f-4f94-b439-443b1161a963 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:01.062778 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd1d78f-ff4f-4f94-b439-443b1161a963 admin admin] PATH_INFO: `/v3/projects/4a8966ff4bda461aa40251ac9740d172` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:01.064641 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd1d78f-ff4f-4f94-b439-443b1161a963 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a8966ff4bda461aa40251ac9740d172)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:01.068333 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd1d78f-ff4f-4f94-b439-443b1161a963 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:01.080133 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f35c7e-a507-4e83-8814-81f3843168dc admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:01.111009 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e34808-d46b-40c3-bf21-1d43ae292c37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:01.118829 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e34808-d46b-40c3-bf21-1d43ae292c37 admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:01.119610 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e34808-d46b-40c3-bf21-1d43ae292c37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:01.120358 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e34808-d46b-40c3-bf21-1d43ae292c37 admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:01.121604 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e34808-d46b-40c3-bf21-1d43ae292c37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:01.133161 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e34808-d46b-40c3-bf21-1d43ae292c37 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:01.179619 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-cbd1d78f-ff4f-4f94-b439-443b1161a963 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a8966ff4bda461aa40251ac9740d172', 'request_id': 'req-cbd1d78f-ff4f-4f94-b439-443b1161a963'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:01.184656 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1261/2535] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:24:00 2022] GET /identity/v3/roles => generated 1420 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 20:24:01.187062 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e34808-d46b-40c3-bf21-1d43ae292c37 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:01.198119 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-cbd1d78f-ff4f-4f94-b439-443b1161a963 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a8966ff4bda461aa40251ac9740d172', 'request_id': 'req-cbd1d78f-ff4f-4f94-b439-443b1161a963'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:01.211979 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e34808-d46b-40c3-bf21-1d43ae292c37 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:01.218323 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-cbd1d78f-ff4f-4f94-b439-443b1161a963 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a8966ff4bda461aa40251ac9740d172', 'request_id': 'req-cbd1d78f-ff4f-4f94-b439-443b1161a963'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:01.269598 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-cbd1d78f-ff4f-4f94-b439-443b1161a963 admin admin] The token cache is being invalidate because project 4a8966ff4bda461aa40251ac9740d172 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81772) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:24:01.270318 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-cbd1d78f-ff4f-4f94-b439-443b1161a963 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:01.271637 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-cbd1d78f-ff4f-4f94-b439-443b1161a963 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:01.284073 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1275/2536] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:24:00 2022] DELETE /identity/v3/projects/4a8966ff4bda461aa40251ac9740d172 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:24:01.286799 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd1d78f-ff4f-4f94-b439-443b1161a963 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:01.290341 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-cbd1d78f-ff4f-4f94-b439-443b1161a963 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:01.321282 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:24:01.405176 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0358dde3-f8f1-4b64-aa6d-ba9c6668feae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:01.406395 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0358dde3-f8f1-4b64-aa6d-ba9c6668feae admin admin] REQUEST_METHOD: `PUT` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:01.406939 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0358dde3-f8f1-4b64-aa6d-ba9c6668feae admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:01.407458 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0358dde3-f8f1-4b64-aa6d-ba9c6668feae admin admin] PATH_INFO: `/v3/projects/be3586699a5a4c6cb27ca651b7854e1f/users/8969fadfef20424bbc28101ca3d7ae06/roles/e0ab41d8ae22474688b5361bcc75e0f0` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:01.411168 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0358dde3-f8f1-4b64-aa6d-ba9c6668feae admin admin] RBAC: Authorizing `identity:create_grant(project_id=be3586699a5a4c6cb27ca651b7854e1f, user_id=8969fadfef20424bbc28101ca3d7ae06, role_id=e0ab41d8ae22474688b5361bcc75e0f0)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:01.416898 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0358dde3-f8f1-4b64-aa6d-ba9c6668feae admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:01.418009 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd1d78f-ff4f-4f94-b439-443b1161a963 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:01.423677 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8751998-dc46-42a8-9864-c12a93b5dfc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:01.424509 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8751998-dc46-42a8-9864-c12a93b5dfc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:01.424789 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8751998-dc46-42a8-9864-c12a93b5dfc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:01.425062 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8751998-dc46-42a8-9864-c12a93b5dfc5 admin admin] PATH_INFO: `/v3/projects/48d7280f7ee04d389193da7bf05eaf1a` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:01.426986 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8751998-dc46-42a8-9864-c12a93b5dfc5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=48d7280f7ee04d389193da7bf05eaf1a)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:01.429255 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8751998-dc46-42a8-9864-c12a93b5dfc5 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:01.445507 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1262/2537] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:24:01 2022] PUT /identity/v3/projects/be3586699a5a4c6cb27ca651b7854e1f/users/8969fadfef20424bbc28101ca3d7ae06/roles/e0ab41d8ae22474688b5361bcc75e0f0 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:24:01.445507 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d51efa6-f134-4d06-91e2-974fd4e60da4 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:01.445507 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d51efa6-f134-4d06-91e2-974fd4e60da4 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:01.445507 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d51efa6-f134-4d06-91e2-974fd4e60da4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:01.485525 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-9d51efa6-f134-4d06-91e2-974fd4e60da4 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:24:01.487856 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-9d51efa6-f134-4d06-91e2-974fd4e60da4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:01.518664 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-a8751998-dc46-42a8-9864-c12a93b5dfc5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48d7280f7ee04d389193da7bf05eaf1a', 'request_id': 'req-a8751998-dc46-42a8-9864-c12a93b5dfc5'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:01.533750 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-a8751998-dc46-42a8-9864-c12a93b5dfc5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48d7280f7ee04d389193da7bf05eaf1a', 'request_id': 'req-a8751998-dc46-42a8-9864-c12a93b5dfc5'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:01.545370 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-a8751998-dc46-42a8-9864-c12a93b5dfc5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48d7280f7ee04d389193da7bf05eaf1a', 'request_id': 'req-a8751998-dc46-42a8-9864-c12a93b5dfc5'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:01.570714 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-a8751998-dc46-42a8-9864-c12a93b5dfc5 admin admin] The token cache is being invalidate because project 48d7280f7ee04d389193da7bf05eaf1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81772) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:24:01.571188 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-a8751998-dc46-42a8-9864-c12a93b5dfc5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:01.571921 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-a8751998-dc46-42a8-9864-c12a93b5dfc5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:01.577137 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1276/2538] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:24:01 2022] DELETE /identity/v3/projects/48d7280f7ee04d389193da7bf05eaf1a => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:24:01.577137 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a69658-289a-4193-b68e-968bf8671e26 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:01.577137 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a69658-289a-4193-b68e-968bf8671e26 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:01.577137 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a69658-289a-4193-b68e-968bf8671e26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:01.605699 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1263/2539] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:24:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:24:01.607095 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d51efa6-f134-4d06-91e2-974fd4e60da4 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:01.609001 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-9d51efa6-f134-4d06-91e2-974fd4e60da4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:01.623786 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-91a69658-289a-4193-b68e-968bf8671e26 None None] MFA Rules not processed for user `8969fadfef20424bbc28101ca3d7ae06`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:24:01.625361 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-91a69658-289a-4193-b68e-968bf8671e26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:01.625676 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d51efa6-f134-4d06-91e2-974fd4e60da4 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:01.724939 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448218ab-62cf-4cf4-afa8-65cc554cb10f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:01.726004 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448218ab-62cf-4cf4-afa8-65cc554cb10f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:01.726473 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448218ab-62cf-4cf4-afa8-65cc554cb10f admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:01.726935 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448218ab-62cf-4cf4-afa8-65cc554cb10f admin admin] PATH_INFO: `/v3/users/6ed33b2c77ed40659aad1827865939c5` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:01.728906 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448218ab-62cf-4cf4-afa8-65cc554cb10f admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ed33b2c77ed40659aad1827865939c5)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:01.732800 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448218ab-62cf-4cf4-afa8-65cc554cb10f admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:01.735912 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:24:01.765728 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1277/2540] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:24:01 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:24:01.765728 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ec8128-bc29-4141-bc7d-d9b2d07b2111 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:01.765728 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ec8128-bc29-4141-bc7d-d9b2d07b2111 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:01.765728 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ec8128-bc29-4141-bc7d-d9b2d07b2111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:01.798855 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-f5ec8128-bc29-4141-bc7d-d9b2d07b2111 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:24:01.800857 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-f5ec8128-bc29-4141-bc7d-d9b2d07b2111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:01.817330 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1278/2541] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:24:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:24:01.833642 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ec8128-bc29-4141-bc7d-d9b2d07b2111 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:01.846856 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-f5ec8128-bc29-4141-bc7d-d9b2d07b2111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:01.849025 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-448218ab-62cf-4cf4-afa8-65cc554cb10f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ed33b2c77ed40659aad1827865939c5'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:01.865598 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-448218ab-62cf-4cf4-afa8-65cc554cb10f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ed33b2c77ed40659aad1827865939c5'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:01.866994 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-448218ab-62cf-4cf4-afa8-65cc554cb10f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ed33b2c77ed40659aad1827865939c5'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:01.867838 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-448218ab-62cf-4cf4-afa8-65cc554cb10f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ed33b2c77ed40659aad1827865939c5'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:01.878763 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-448218ab-62cf-4cf4-afa8-65cc554cb10f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ed33b2c77ed40659aad1827865939c5'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:01.918794 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1264/2542] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:24:01 2022] DELETE /identity/v3/users/6ed33b2c77ed40659aad1827865939c5 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:24:01.920619 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:24:01.921564 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448218ab-62cf-4cf4-afa8-65cc554cb10f admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:01.926828 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-448218ab-62cf-4cf4-afa8-65cc554cb10f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:02.024616 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ec8128-bc29-4141-bc7d-d9b2d07b2111 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:02.056788 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448218ab-62cf-4cf4-afa8-65cc554cb10f admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:02.063947 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a630a175-6730-4a01-bd62-ae05f178eccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:02.064903 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a630a175-6730-4a01-bd62-ae05f178eccf admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:02.064903 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a630a175-6730-4a01-bd62-ae05f178eccf admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:02.065360 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a630a175-6730-4a01-bd62-ae05f178eccf admin admin] PATH_INFO: `/v3/domains` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:02.065918 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a630a175-6730-4a01-bd62-ae05f178eccf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:24:02.066271 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a630a175-6730-4a01-bd62-ae05f178eccf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:02.068020 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a630a175-6730-4a01-bd62-ae05f178eccf admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:02.092663 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1265/2543] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 6 20:24:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:24:02.096651 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a630a175-6730-4a01-bd62-ae05f178eccf admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:02.132153 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a630a175-6730-4a01-bd62-ae05f178eccf admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:02.142980 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6936e2f0-582b-4d60-8961-bac968617de0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:02.145942 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6936e2f0-582b-4d60-8961-bac968617de0 admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:02.146414 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6936e2f0-582b-4d60-8961-bac968617de0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:02.146900 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6936e2f0-582b-4d60-8961-bac968617de0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:02.148191 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6936e2f0-582b-4d60-8961-bac968617de0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:02.152317 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6936e2f0-582b-4d60-8961-bac968617de0 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:02.187709 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e11f58-6618-4379-834b-5a9c4ac0a7bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df1bca993bc142908cc7cf85eabd2b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:02.190205 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e11f58-6618-4379-834b-5a9c4ac0a7bb service cinder] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:02.190716 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e11f58-6618-4379-834b-5a9c4ac0a7bb service cinder] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:02.191649 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e11f58-6618-4379-834b-5a9c4ac0a7bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:02.199778 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-00e11f58-6618-4379-834b-5a9c4ac0a7bb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:02.229257 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1266/2544] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:24:02 2022] POST /identity/v3/projects => generated 385 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:24:02.232119 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6936e2f0-582b-4d60-8961-bac968617de0 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:02.338328 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e11f58-6618-4379-834b-5a9c4ac0a7bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:02.340364 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e11f58-6618-4379-834b-5a9c4ac0a7bb service cinder] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:02.367934 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1279/2545] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:24:01 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 550 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:24:02.380915 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:24:02.411270 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6936e2f0-582b-4d60-8961-bac968617de0 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:02.419582 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3291ab86-cdf4-4c32-9a91-0dff56e8dbad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:02.421543 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3291ab86-cdf4-4c32-9a91-0dff56e8dbad admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:02.422749 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3291ab86-cdf4-4c32-9a91-0dff56e8dbad admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:02.422993 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3291ab86-cdf4-4c32-9a91-0dff56e8dbad admin admin] PATH_INFO: `/v3/users` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:02.425700 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3291ab86-cdf4-4c32-9a91-0dff56e8dbad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:02.431241 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3291ab86-cdf4-4c32-9a91-0dff56e8dbad admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:02.444170 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e11f58-6618-4379-834b-5a9c4ac0a7bb service cinder] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:02.447078 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-00e11f58-6618-4379-834b-5a9c4ac0a7bb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:02.492500 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1267/2546] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:24:02 2022] POST /identity/v3/users => generated 293 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:24:02.499876 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3291ab86-cdf4-4c32-9a91-0dff56e8dbad admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:02.520647 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e11f58-6618-4379-834b-5a9c4ac0a7bb service cinder] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:02.541442 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3be3f84-044c-4904-96c9-e15f828c5d80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:02.541736 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3291ab86-cdf4-4c32-9a91-0dff56e8dbad admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:02.546012 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3be3f84-044c-4904-96c9-e15f828c5d80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:02.546700 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3be3f84-044c-4904-96c9-e15f828c5d80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:02.547006 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3be3f84-044c-4904-96c9-e15f828c5d80 admin admin] PATH_INFO: `/v3/projects/94f1527ff6a24463b13a1936a5ac57cd` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:02.560211 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3be3f84-044c-4904-96c9-e15f828c5d80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=94f1527ff6a24463b13a1936a5ac57cd)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:02.560211 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3be3f84-044c-4904-96c9-e15f828c5d80 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:02.560807 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bba4f95-9f44-4a5f-85c4-f7c683668b69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:02.561396 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bba4f95-9f44-4a5f-85c4-f7c683668b69 admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:02.562529 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bba4f95-9f44-4a5f-85c4-f7c683668b69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:02.570259 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bba4f95-9f44-4a5f-85c4-f7c683668b69 admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:02.574140 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bba4f95-9f44-4a5f-85c4-f7c683668b69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:02.577139 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bba4f95-9f44-4a5f-85c4-f7c683668b69 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:02.609802 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1268/2547] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:24:02 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:24:02.609802 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bba4f95-9f44-4a5f-85c4-f7c683668b69 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:02.612392 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-3bba4f95-9f44-4a5f-85c4-f7c683668b69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:02.679188 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-f3be3f84-044c-4904-96c9-e15f828c5d80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94f1527ff6a24463b13a1936a5ac57cd', 'request_id': 'req-f3be3f84-044c-4904-96c9-e15f828c5d80'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:02.698890 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-f3be3f84-044c-4904-96c9-e15f828c5d80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94f1527ff6a24463b13a1936a5ac57cd', 'request_id': 'req-f3be3f84-044c-4904-96c9-e15f828c5d80'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:02.713840 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-f3be3f84-044c-4904-96c9-e15f828c5d80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94f1527ff6a24463b13a1936a5ac57cd', 'request_id': 'req-f3be3f84-044c-4904-96c9-e15f828c5d80'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:02.740672 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-f3be3f84-044c-4904-96c9-e15f828c5d80 admin admin] The token cache is being invalidate because project 94f1527ff6a24463b13a1936a5ac57cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81772) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:24:02.741187 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-f3be3f84-044c-4904-96c9-e15f828c5d80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:02.742745 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-f3be3f84-044c-4904-96c9-e15f828c5d80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:02.746210 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1280/2548] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:24:02 2022] DELETE /identity/v3/projects/94f1527ff6a24463b13a1936a5ac57cd => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 20:24:02.748834 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3be3f84-044c-4904-96c9-e15f828c5d80 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:02.751774 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-f3be3f84-044c-4904-96c9-e15f828c5d80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:02.754685 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bba4f95-9f44-4a5f-85c4-f7c683668b69 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:02.817262 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:24:02.935575 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532acd26-23da-4bf1-ad44-8dd8b1b473b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df1bca993bc142908cc7cf85eabd2b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:02.936919 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532acd26-23da-4bf1-ad44-8dd8b1b473b8 service cinder] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:02.937412 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532acd26-23da-4bf1-ad44-8dd8b1b473b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:02.937903 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532acd26-23da-4bf1-ad44-8dd8b1b473b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:02.940286 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-532acd26-23da-4bf1-ad44-8dd8b1b473b8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:03.002302 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3be3f84-044c-4904-96c9-e15f828c5d80 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:03.017084 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af27db6-8d7f-442c-a515-358c234bb78e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:03.018140 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af27db6-8d7f-442c-a515-358c234bb78e admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:03.018565 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af27db6-8d7f-442c-a515-358c234bb78e admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:03.019168 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af27db6-8d7f-442c-a515-358c234bb78e admin admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:03.030721 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af27db6-8d7f-442c-a515-358c234bb78e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:03.033798 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af27db6-8d7f-442c-a515-358c234bb78e admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:03.057995 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532acd26-23da-4bf1-ad44-8dd8b1b473b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:03.062221 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1281/2549] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:24:02 2022] GET /identity/v3/roles => generated 1420 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:24:03.063893 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532acd26-23da-4bf1-ad44-8dd8b1b473b8 service cinder] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:03.069930 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af27db6-8d7f-442c-a515-358c234bb78e admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:03.095493 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1269/2550] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:24:02 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 489 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:24:03.115368 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af27db6-8d7f-442c-a515-358c234bb78e admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:03.133083 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf9582d-ee5f-4658-a5c2-d13c77b472aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:03.134189 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf9582d-ee5f-4658-a5c2-d13c77b472aa admin admin] REQUEST_METHOD: `PUT` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:03.134649 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf9582d-ee5f-4658-a5c2-d13c77b472aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:03.135103 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf9582d-ee5f-4658-a5c2-d13c77b472aa admin admin] PATH_INFO: `/v3/projects/121f31a3391b43948e3eeec3f2332dd1/users/dc49853bd7f94f8e915835c465f6fc20/roles/4eea39e24b06431aa357053fe7907a3d` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:03.196210 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf9582d-ee5f-4658-a5c2-d13c77b472aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=121f31a3391b43948e3eeec3f2332dd1, user_id=dc49853bd7f94f8e915835c465f6fc20, role_id=4eea39e24b06431aa357053fe7907a3d)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:03.202347 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf9582d-ee5f-4658-a5c2-d13c77b472aa admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:03.221727 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1282/2551] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:24:03 2022] PUT /identity/v3/projects/121f31a3391b43948e3eeec3f2332dd1/users/dc49853bd7f94f8e915835c465f6fc20/roles/4eea39e24b06431aa357053fe7907a3d => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:24:03.246163 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bec6c51-a3ec-4d2a-9922-1dd33984252e None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:03.246652 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bec6c51-a3ec-4d2a-9922-1dd33984252e None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:03.247143 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bec6c51-a3ec-4d2a-9922-1dd33984252e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:03.299458 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-4bec6c51-a3ec-4d2a-9922-1dd33984252e None None] MFA Rules not processed for user `dc49853bd7f94f8e915835c465f6fc20`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:24:03.302404 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-4bec6c51-a3ec-4d2a-9922-1dd33984252e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:03.373544 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:24:03.516231 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1270/2552] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:24:03 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:24:03.517259 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf9582d-ee5f-4658-a5c2-d13c77b472aa admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:03.519936 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-8bf9582d-ee5f-4658-a5c2-d13c77b472aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:03.643668 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf9582d-ee5f-4658-a5c2-d13c77b472aa admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:03.662228 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8d92e3-5d8c-4f0b-bd61-792f987227c3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5776cbf4f6714b2083c3ebdd6d98cc87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:03.664112 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8d92e3-5d8c-4f0b-bd61-792f987227c3 service nova] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:03.664834 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8d92e3-5d8c-4f0b-bd61-792f987227c3 service nova] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:03.665661 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8d92e3-5d8c-4f0b-bd61-792f987227c3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:03.692581 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8d92e3-5d8c-4f0b-bd61-792f987227c3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:03.696044 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8d92e3-5d8c-4f0b-bd61-792f987227c3 service nova] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:03.719277 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1283/2553] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:24:03 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 20:24:09.986306 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bec6c51-a3ec-4d2a-9922-1dd33984252e None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:09.988986 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-4bec6c51-a3ec-4d2a-9922-1dd33984252e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:10.114567 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bec6c51-a3ec-4d2a-9922-1dd33984252e None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:10.122166 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c654bb82-9793-44f8-a8ff-900f1ec806f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:10.123301 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c654bb82-9793-44f8-a8ff-900f1ec806f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:10.123746 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c654bb82-9793-44f8-a8ff-900f1ec806f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:10.124203 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c654bb82-9793-44f8-a8ff-900f1ec806f4 admin admin] PATH_INFO: `/v3/users/f00e10c87983498dbab6011a41f5e1da` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:10.126609 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c654bb82-9793-44f8-a8ff-900f1ec806f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f00e10c87983498dbab6011a41f5e1da)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:10.130575 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c654bb82-9793-44f8-a8ff-900f1ec806f4 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:10.208881 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-c654bb82-9793-44f8-a8ff-900f1ec806f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f00e10c87983498dbab6011a41f5e1da'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:10.221526 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-c654bb82-9793-44f8-a8ff-900f1ec806f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f00e10c87983498dbab6011a41f5e1da'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:10.222311 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-c654bb82-9793-44f8-a8ff-900f1ec806f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f00e10c87983498dbab6011a41f5e1da'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:10.223145 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-c654bb82-9793-44f8-a8ff-900f1ec806f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f00e10c87983498dbab6011a41f5e1da'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:10.234174 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-c654bb82-9793-44f8-a8ff-900f1ec806f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f00e10c87983498dbab6011a41f5e1da'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:10.271277 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1271/2554] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:24:09 2022] DELETE /identity/v3/users/f00e10c87983498dbab6011a41f5e1da => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:24:10.277337 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbcb191-a768-4fd7-a072-be0079afd1c2 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:10.278061 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbcb191-a768-4fd7-a072-be0079afd1c2 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:10.278404 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbcb191-a768-4fd7-a072-be0079afd1c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:10.307963 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-bcbcb191-a768-4fd7-a072-be0079afd1c2 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:24:10.310199 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-bcbcb191-a768-4fd7-a072-be0079afd1c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:10.404664 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1284/2555] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:24:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:24:10.414682 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c654bb82-9793-44f8-a8ff-900f1ec806f4 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:10.418092 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-c654bb82-9793-44f8-a8ff-900f1ec806f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:10.525456 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:24:10.531214 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c654bb82-9793-44f8-a8ff-900f1ec806f4 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:10.537444 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3216d0b7-3d1a-4557-b3e4-7ee50b6ef363 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:10.538937 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3216d0b7-3d1a-4557-b3e4-7ee50b6ef363 service neutron] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:10.539381 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3216d0b7-3d1a-4557-b3e4-7ee50b6ef363 service neutron] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:10.539798 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3216d0b7-3d1a-4557-b3e4-7ee50b6ef363 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:10.553270 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3216d0b7-3d1a-4557-b3e4-7ee50b6ef363 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:10.554851 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3216d0b7-3d1a-4557-b3e4-7ee50b6ef363 service neutron] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:10.566798 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1272/2556] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:24:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:24:10.944505 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbcb191-a768-4fd7-a072-be0079afd1c2 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:10.946131 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-bcbcb191-a768-4fd7-a072-be0079afd1c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:10.959606 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbcb191-a768-4fd7-a072-be0079afd1c2 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:10.965694 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a52755-a1fc-438a-a313-3350e708d38f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:10.966525 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a52755-a1fc-438a-a313-3350e708d38f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:10.967071 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a52755-a1fc-438a-a313-3350e708d38f admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:10.967440 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a52755-a1fc-438a-a313-3350e708d38f admin admin] PATH_INFO: `/v3/projects/8683e525130f41479467d887f438da8e` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:10.969298 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a52755-a1fc-438a-a313-3350e708d38f admin admin] RBAC: Authorizing `identity:delete_project(project_id=8683e525130f41479467d887f438da8e)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:10.972109 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a52755-a1fc-438a-a313-3350e708d38f admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:10.975577 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:24:11.057175 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-86a52755-a1fc-438a-a313-3350e708d38f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8683e525130f41479467d887f438da8e', 'request_id': 'req-86a52755-a1fc-438a-a313-3350e708d38f'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:11.069451 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-86a52755-a1fc-438a-a313-3350e708d38f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8683e525130f41479467d887f438da8e', 'request_id': 'req-86a52755-a1fc-438a-a313-3350e708d38f'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:11.081720 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-86a52755-a1fc-438a-a313-3350e708d38f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8683e525130f41479467d887f438da8e', 'request_id': 'req-86a52755-a1fc-438a-a313-3350e708d38f'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:11.097030 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-86a52755-a1fc-438a-a313-3350e708d38f admin admin] The token cache is being invalidate because project 8683e525130f41479467d887f438da8e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81772) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:24:11.097397 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-86a52755-a1fc-438a-a313-3350e708d38f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:11.098203 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-86a52755-a1fc-438a-a313-3350e708d38f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:11.100248 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1285/2557] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:24:10 2022] DELETE /identity/v3/projects/8683e525130f41479467d887f438da8e => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:24:11.105904 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3216d0b7-3d1a-4557-b3e4-7ee50b6ef363 service neutron] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:11.124744 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3216d0b7-3d1a-4557-b3e4-7ee50b6ef363 service neutron] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:11.212817 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc290ca7-cf06-40ab-8faa-35b43ff24f2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:11.213569 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc290ca7-cf06-40ab-8faa-35b43ff24f2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:11.214007 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc290ca7-cf06-40ab-8faa-35b43ff24f2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:11.214359 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc290ca7-cf06-40ab-8faa-35b43ff24f2d admin admin] PATH_INFO: `/v3/users/8969fadfef20424bbc28101ca3d7ae06` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:11.215809 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc290ca7-cf06-40ab-8faa-35b43ff24f2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=8969fadfef20424bbc28101ca3d7ae06)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:11.218066 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc290ca7-cf06-40ab-8faa-35b43ff24f2d admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:11.286620 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-fc290ca7-cf06-40ab-8faa-35b43ff24f2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8969fadfef20424bbc28101ca3d7ae06'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:11.296975 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-fc290ca7-cf06-40ab-8faa-35b43ff24f2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8969fadfef20424bbc28101ca3d7ae06'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:11.297754 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-fc290ca7-cf06-40ab-8faa-35b43ff24f2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8969fadfef20424bbc28101ca3d7ae06'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:11.298437 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-fc290ca7-cf06-40ab-8faa-35b43ff24f2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8969fadfef20424bbc28101ca3d7ae06'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:11.309073 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-fc290ca7-cf06-40ab-8faa-35b43ff24f2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8969fadfef20424bbc28101ca3d7ae06'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:11.333146 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1273/2558] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:24:11 2022] DELETE /identity/v3/users/8969fadfef20424bbc28101ca3d7ae06 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:24:11.637766 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a52755-a1fc-438a-a313-3350e708d38f admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:11.638367 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-86a52755-a1fc-438a-a313-3350e708d38f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:11.655657 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a52755-a1fc-438a-a313-3350e708d38f admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:11.746199 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc290ca7-cf06-40ab-8faa-35b43ff24f2d admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:11.746458 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6c63d7-23d8-45f3-b333-bde2f8ad3b36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:11.747338 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6c63d7-23d8-45f3-b333-bde2f8ad3b36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:11.747746 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6c63d7-23d8-45f3-b333-bde2f8ad3b36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:11.747913 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-fc290ca7-cf06-40ab-8faa-35b43ff24f2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:11.748222 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6c63d7-23d8-45f3-b333-bde2f8ad3b36 admin admin] PATH_INFO: `/v3/users/dc49853bd7f94f8e915835c465f6fc20` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:11.750310 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6c63d7-23d8-45f3-b333-bde2f8ad3b36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc49853bd7f94f8e915835c465f6fc20)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:11.755654 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6c63d7-23d8-45f3-b333-bde2f8ad3b36 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:11.773956 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc290ca7-cf06-40ab-8faa-35b43ff24f2d admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:11.783126 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b39a7b-451a-4519-919e-e90f88357b6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:11.783762 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b39a7b-451a-4519-919e-e90f88357b6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:11.784258 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b39a7b-451a-4519-919e-e90f88357b6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:11.784849 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b39a7b-451a-4519-919e-e90f88357b6c admin admin] PATH_INFO: `/v3/projects/be3586699a5a4c6cb27ca651b7854e1f` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:11.787543 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b39a7b-451a-4519-919e-e90f88357b6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=be3586699a5a4c6cb27ca651b7854e1f)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:11.791518 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b39a7b-451a-4519-919e-e90f88357b6c admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:11.854304 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-dc6c63d7-23d8-45f3-b333-bde2f8ad3b36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc49853bd7f94f8e915835c465f6fc20'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:11.854990 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-dc6c63d7-23d8-45f3-b333-bde2f8ad3b36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc49853bd7f94f8e915835c465f6fc20'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:11.873010 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-dc6c63d7-23d8-45f3-b333-bde2f8ad3b36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc49853bd7f94f8e915835c465f6fc20'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:11.873817 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-dc6c63d7-23d8-45f3-b333-bde2f8ad3b36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc49853bd7f94f8e915835c465f6fc20'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:11.877008 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-34b39a7b-451a-4519-919e-e90f88357b6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be3586699a5a4c6cb27ca651b7854e1f', 'request_id': 'req-34b39a7b-451a-4519-919e-e90f88357b6c'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:11.883755 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-dc6c63d7-23d8-45f3-b333-bde2f8ad3b36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc49853bd7f94f8e915835c465f6fc20'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:11.891220 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-34b39a7b-451a-4519-919e-e90f88357b6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be3586699a5a4c6cb27ca651b7854e1f', 'request_id': 'req-34b39a7b-451a-4519-919e-e90f88357b6c'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:11.903106 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1286/2559] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:24:11 2022] DELETE /identity/v3/users/dc49853bd7f94f8e915835c465f6fc20 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:24:11.906218 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-34b39a7b-451a-4519-919e-e90f88357b6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be3586699a5a4c6cb27ca651b7854e1f', 'request_id': 'req-34b39a7b-451a-4519-919e-e90f88357b6c'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:11.909589 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb27c0ec-bc8f-4c75-8d5e-e905adbb4b51 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:11.910222 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb27c0ec-bc8f-4c75-8d5e-e905adbb4b51 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:11.911371 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb27c0ec-bc8f-4c75-8d5e-e905adbb4b51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:11.914981 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:24:11.928100 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-34b39a7b-451a-4519-919e-e90f88357b6c admin admin] The token cache is being invalidate because project be3586699a5a4c6cb27ca651b7854e1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81771) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:24:11.928470 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-34b39a7b-451a-4519-919e-e90f88357b6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:11.929291 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-34b39a7b-451a-4519-919e-e90f88357b6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:11.931627 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1274/2560] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:24:11 2022] DELETE /identity/v3/projects/be3586699a5a4c6cb27ca651b7854e1f => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 20:24:11.937832 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c675a47-4ca2-4dce-ab14-d1e9aa00c786 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:11.938392 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c675a47-4ca2-4dce-ab14-d1e9aa00c786 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:11.939131 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c675a47-4ca2-4dce-ab14-d1e9aa00c786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:11.947968 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-fb27c0ec-bc8f-4c75-8d5e-e905adbb4b51 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:24:11.950332 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-fb27c0ec-bc8f-4c75-8d5e-e905adbb4b51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:11.960264 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:24:11.976859 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-2c675a47-4ca2-4dce-ab14-d1e9aa00c786 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:24:11.979156 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-2c675a47-4ca2-4dce-ab14-d1e9aa00c786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:12.058329 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1287/2561] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:24:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:24:12.071804 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb27c0ec-bc8f-4c75-8d5e-e905adbb4b51 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:12.074045 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-fb27c0ec-bc8f-4c75-8d5e-e905adbb4b51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:12.084943 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1275/2562] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:24:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:24:12.102116 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92a5d44-0f5f-40af-8a10-6f524d02234c None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:12.105697 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92a5d44-0f5f-40af-8a10-6f524d02234c None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:12.106210 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92a5d44-0f5f-40af-8a10-6f524d02234c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:12.140285 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-d92a5d44-0f5f-40af-8a10-6f524d02234c None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:24:12.142491 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-d92a5d44-0f5f-40af-8a10-6f524d02234c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:12.157806 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1276/2563] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:24:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:24:12.163238 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb27c0ec-bc8f-4c75-8d5e-e905adbb4b51 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:12.165637 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92a5d44-0f5f-40af-8a10-6f524d02234c None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:12.167828 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af63537b-3ac7-4f14-a514-b5bd6e76b0c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:12.168502 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af63537b-3ac7-4f14-a514-b5bd6e76b0c2 service neutron] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:12.168781 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af63537b-3ac7-4f14-a514-b5bd6e76b0c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:12.169053 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af63537b-3ac7-4f14-a514-b5bd6e76b0c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:12.183205 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af63537b-3ac7-4f14-a514-b5bd6e76b0c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:12.183488 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92a5d44-0f5f-40af-8a10-6f524d02234c None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:12.185879 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af63537b-3ac7-4f14-a514-b5bd6e76b0c2 service neutron] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:12.191369 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b09787f-9edd-4b6b-919c-f482af2bc446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:12.192345 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b09787f-9edd-4b6b-919c-f482af2bc446 admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:12.192785 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b09787f-9edd-4b6b-919c-f482af2bc446 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:12.193208 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b09787f-9edd-4b6b-919c-f482af2bc446 admin admin] PATH_INFO: `/v3/domains` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:12.194078 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b09787f-9edd-4b6b-919c-f482af2bc446 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:24:12.194497 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b09787f-9edd-4b6b-919c-f482af2bc446 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:12.196379 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b09787f-9edd-4b6b-919c-f482af2bc446 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:12.201867 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1288/2564] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:24:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 20:24:12.217430 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1277/2565] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 6 20:24:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 20:24:12.224413 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af63537b-3ac7-4f14-a514-b5bd6e76b0c2 service neutron] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:12.243837 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af63537b-3ac7-4f14-a514-b5bd6e76b0c2 service neutron] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:12.260559 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4da6d3-22ae-4604-8930-1e78a173f2c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:12.261722 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4da6d3-22ae-4604-8930-1e78a173f2c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:12.262223 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4da6d3-22ae-4604-8930-1e78a173f2c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:12.262712 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4da6d3-22ae-4604-8930-1e78a173f2c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:12.263914 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4da6d3-22ae-4604-8930-1e78a173f2c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:12.267858 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4da6d3-22ae-4604-8930-1e78a173f2c5 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:12.319591 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1289/2566] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:24:12 2022] POST /identity/v3/projects => generated 407 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:24:12.325485 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b09787f-9edd-4b6b-919c-f482af2bc446 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:12.344181 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b09787f-9edd-4b6b-919c-f482af2bc446 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:12.467991 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e805a122-bb75-425b-aa6a-f1e63d14fd97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:12.471703 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e805a122-bb75-425b-aa6a-f1e63d14fd97 admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:12.471703 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e805a122-bb75-425b-aa6a-f1e63d14fd97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:12.471703 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e805a122-bb75-425b-aa6a-f1e63d14fd97 admin admin] PATH_INFO: `/v3/users` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:12.471703 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e805a122-bb75-425b-aa6a-f1e63d14fd97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:12.475547 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e805a122-bb75-425b-aa6a-f1e63d14fd97 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:12.515508 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1278/2567] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:24:12 2022] POST /identity/v3/users => generated 304 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:24:12.519820 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4da6d3-22ae-4604-8930-1e78a173f2c5 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:12.536678 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4da6d3-22ae-4604-8930-1e78a173f2c5 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:12.542443 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2abbee1-6919-4b91-a9b5-f37473f312ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:12.543079 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2abbee1-6919-4b91-a9b5-f37473f312ae admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:12.543349 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2abbee1-6919-4b91-a9b5-f37473f312ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:12.543629 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2abbee1-6919-4b91-a9b5-f37473f312ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:12.544445 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2abbee1-6919-4b91-a9b5-f37473f312ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:12.546296 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2abbee1-6919-4b91-a9b5-f37473f312ae admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:12.558942 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1290/2568] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:24:12 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:24:12.564435 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e805a122-bb75-425b-aa6a-f1e63d14fd97 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:12.584528 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e805a122-bb75-425b-aa6a-f1e63d14fd97 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:12.592126 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9608ce-2889-41a7-92ee-ce55d1edbe21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:12.593443 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9608ce-2889-41a7-92ee-ce55d1edbe21 admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:12.593966 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9608ce-2889-41a7-92ee-ce55d1edbe21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:12.594452 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9608ce-2889-41a7-92ee-ce55d1edbe21 admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:12.595689 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9608ce-2889-41a7-92ee-ce55d1edbe21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:12.598983 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9608ce-2889-41a7-92ee-ce55d1edbe21 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:12.628338 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1279/2569] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:24:12 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:24:12.629840 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2abbee1-6919-4b91-a9b5-f37473f312ae admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:12.657043 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2abbee1-6919-4b91-a9b5-f37473f312ae admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:12.665113 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01631e71-c1b3-4955-8de7-630e16ba6143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:12.666171 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01631e71-c1b3-4955-8de7-630e16ba6143 admin admin] REQUEST_METHOD: `PUT` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:12.666621 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01631e71-c1b3-4955-8de7-630e16ba6143 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:12.667093 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01631e71-c1b3-4955-8de7-630e16ba6143 admin admin] PATH_INFO: `/v3/projects/840ba46c70e3484c830bdc4b5da5020a/users/154e87a9077f4738995c62beaddcafe2/roles/4eea39e24b06431aa357053fe7907a3d` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:12.710216 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01631e71-c1b3-4955-8de7-630e16ba6143 admin admin] RBAC: Authorizing `identity:create_grant(project_id=840ba46c70e3484c830bdc4b5da5020a, user_id=154e87a9077f4738995c62beaddcafe2, role_id=4eea39e24b06431aa357053fe7907a3d)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:12.712565 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9608ce-2889-41a7-92ee-ce55d1edbe21 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:12.715520 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-dd9608ce-2889-41a7-92ee-ce55d1edbe21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:12.716739 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01631e71-c1b3-4955-8de7-630e16ba6143 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:12.744234 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1291/2570] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:24:12 2022] PUT /identity/v3/projects/840ba46c70e3484c830bdc4b5da5020a/users/154e87a9077f4738995c62beaddcafe2/roles/4eea39e24b06431aa357053fe7907a3d => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 20:24:12.761317 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9608ce-2889-41a7-92ee-ce55d1edbe21 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:12.764061 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ef2f0f-39f4-45a6-8b97-0ba47d80e72b None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:12.764518 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ef2f0f-39f4-45a6-8b97-0ba47d80e72b None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:12.765014 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ef2f0f-39f4-45a6-8b97-0ba47d80e72b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:12.824421 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-d0ef2f0f-39f4-45a6-8b97-0ba47d80e72b None None] MFA Rules not processed for user `154e87a9077f4738995c62beaddcafe2`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:24:12.826526 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-d0ef2f0f-39f4-45a6-8b97-0ba47d80e72b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:12.870335 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:24:12.888847 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bf3332-7881-4967-834d-b26ebce3d6bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:12.890505 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bf3332-7881-4967-834d-b26ebce3d6bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:12.891295 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bf3332-7881-4967-834d-b26ebce3d6bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:12.892217 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bf3332-7881-4967-834d-b26ebce3d6bd admin admin] PATH_INFO: `/v3/projects/121f31a3391b43948e3eeec3f2332dd1` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:12.894116 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bf3332-7881-4967-834d-b26ebce3d6bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=121f31a3391b43948e3eeec3f2332dd1)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:12.897447 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bf3332-7881-4967-834d-b26ebce3d6bd admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:12.899564 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:24:12.943784 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1292/2571] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:24:12 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:24:12.949893 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ef2f0f-39f4-45a6-8b97-0ba47d80e72b None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:12.964170 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ef2f0f-39f4-45a6-8b97-0ba47d80e72b None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:12.968431 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-e4bf3332-7881-4967-834d-b26ebce3d6bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '121f31a3391b43948e3eeec3f2332dd1', 'request_id': 'req-e4bf3332-7881-4967-834d-b26ebce3d6bd'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:12.982236 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-e4bf3332-7881-4967-834d-b26ebce3d6bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '121f31a3391b43948e3eeec3f2332dd1', 'request_id': 'req-e4bf3332-7881-4967-834d-b26ebce3d6bd'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:12.995143 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-e4bf3332-7881-4967-834d-b26ebce3d6bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '121f31a3391b43948e3eeec3f2332dd1', 'request_id': 'req-e4bf3332-7881-4967-834d-b26ebce3d6bd'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:13.015641 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-e4bf3332-7881-4967-834d-b26ebce3d6bd admin admin] The token cache is being invalidate because project 121f31a3391b43948e3eeec3f2332dd1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81771) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:24:13.016139 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-e4bf3332-7881-4967-834d-b26ebce3d6bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:13.018123 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-e4bf3332-7881-4967-834d-b26ebce3d6bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:13.025750 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1280/2572] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:24:12 2022] DELETE /identity/v3/projects/121f31a3391b43948e3eeec3f2332dd1 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:24:13.028992 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132d47ec-b0ea-4c70-9b2c-6de9f120d198 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:13.030230 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132d47ec-b0ea-4c70-9b2c-6de9f120d198 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:13.030230 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132d47ec-b0ea-4c70-9b2c-6de9f120d198 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:13.048959 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f497844b-60a4-4f20-a688-240f64421031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:13.049673 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f497844b-60a4-4f20-a688-240f64421031 admin admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:13.049971 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f497844b-60a4-4f20-a688-240f64421031 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:13.050229 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f497844b-60a4-4f20-a688-240f64421031 admin admin] PATH_INFO: `/v3/projects` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:13.051095 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f497844b-60a4-4f20-a688-240f64421031 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:13.053565 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f497844b-60a4-4f20-a688-240f64421031 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:13.071209 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-132d47ec-b0ea-4c70-9b2c-6de9f120d198 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:24:13.072970 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-132d47ec-b0ea-4c70-9b2c-6de9f120d198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:13.085878 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1281/2573] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:24:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:24:13.098168 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5173802-aecd-41ea-93fb-9f504c3b9d6a None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:13.098780 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5173802-aecd-41ea-93fb-9f504c3b9d6a None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:13.099085 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5173802-aecd-41ea-93fb-9f504c3b9d6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:13.114609 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1293/2574] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:24:12 2022] POST /identity/v3/projects => generated 409 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 20:24:13.121086 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f497844b-60a4-4f20-a688-240f64421031 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:13.125075 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-f497844b-60a4-4f20-a688-240f64421031 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:13.130668 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-a5173802-aecd-41ea-93fb-9f504c3b9d6a None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:24:13.132528 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-a5173802-aecd-41ea-93fb-9f504c3b9d6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:13.251214 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1282/2575] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:24:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:24:13.252583 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5173802-aecd-41ea-93fb-9f504c3b9d6a None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:13.267572 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5173802-aecd-41ea-93fb-9f504c3b9d6a None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:13.272210 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f497844b-60a4-4f20-a688-240f64421031 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:13.275938 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31c463b-42b3-451d-8234-1b435a16685d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:13.277265 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31c463b-42b3-451d-8234-1b435a16685d admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:13.277817 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31c463b-42b3-451d-8234-1b435a16685d admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:13.278295 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31c463b-42b3-451d-8234-1b435a16685d admin admin] PATH_INFO: `/v3/domains` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:13.279482 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31c463b-42b3-451d-8234-1b435a16685d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:24:13.280029 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31c463b-42b3-451d-8234-1b435a16685d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:13.280170 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917b56de-8394-48f2-ac5a-671fab3a0cd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:13.282657 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917b56de-8394-48f2-ac5a-671fab3a0cd3 admin admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:13.282657 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917b56de-8394-48f2-ac5a-671fab3a0cd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:13.282657 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917b56de-8394-48f2-ac5a-671fab3a0cd3 admin admin] PATH_INFO: `/v3/users` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:13.282657 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917b56de-8394-48f2-ac5a-671fab3a0cd3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:13.283255 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31c463b-42b3-451d-8234-1b435a16685d admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:13.284679 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917b56de-8394-48f2-ac5a-671fab3a0cd3 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:13.309156 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1283/2576] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 6 20:24:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:24:13.316189 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31c463b-42b3-451d-8234-1b435a16685d admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:13.317509 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1294/2577] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:24:13 2022] POST /identity/v3/users => generated 311 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:24:13.322987 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917b56de-8394-48f2-ac5a-671fab3a0cd3 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:13.337047 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917b56de-8394-48f2-ac5a-671fab3a0cd3 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:13.337855 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31c463b-42b3-451d-8234-1b435a16685d admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:13.342996 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb296e62-f6b8-4152-9dea-63583402eb1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:13.343627 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb296e62-f6b8-4152-9dea-63583402eb1c admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:13.343973 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb296e62-f6b8-4152-9dea-63583402eb1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:13.344267 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb296e62-f6b8-4152-9dea-63583402eb1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:13.345408 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb296e62-f6b8-4152-9dea-63583402eb1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:13.346974 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb296e62-f6b8-4152-9dea-63583402eb1c admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:13.347367 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b57106d-89b8-409c-bb3c-0f9a7ceb03bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:13.348036 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b57106d-89b8-409c-bb3c-0f9a7ceb03bf admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:13.348714 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b57106d-89b8-409c-bb3c-0f9a7ceb03bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:13.349042 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b57106d-89b8-409c-bb3c-0f9a7ceb03bf admin admin] PATH_INFO: `/v3/projects` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:13.350201 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b57106d-89b8-409c-bb3c-0f9a7ceb03bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:13.354098 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b57106d-89b8-409c-bb3c-0f9a7ceb03bf admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:13.362131 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1295/2578] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:24:13 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:24:13.367802 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb296e62-f6b8-4152-9dea-63583402eb1c admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:13.381189 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb296e62-f6b8-4152-9dea-63583402eb1c admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:13.388221 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7b2cdf-ec64-4cdb-839a-dab0fd42811f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:13.389034 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7b2cdf-ec64-4cdb-839a-dab0fd42811f admin admin] REQUEST_METHOD: `PUT` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:13.389389 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7b2cdf-ec64-4cdb-839a-dab0fd42811f admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:13.389864 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7b2cdf-ec64-4cdb-839a-dab0fd42811f admin admin] PATH_INFO: `/v3/projects/3e4444b1ae8340c4b07216158d2fc761/users/66a7634943ea49fc9b571b7b6acad938/roles/e0ab41d8ae22474688b5361bcc75e0f0` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:13.395757 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1284/2579] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:24:13 2022] POST /identity/v3/projects => generated 373 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:24:13.400393 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b57106d-89b8-409c-bb3c-0f9a7ceb03bf admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:13.415095 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7b2cdf-ec64-4cdb-839a-dab0fd42811f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e4444b1ae8340c4b07216158d2fc761, user_id=66a7634943ea49fc9b571b7b6acad938, role_id=e0ab41d8ae22474688b5361bcc75e0f0)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:13.420962 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7b2cdf-ec64-4cdb-839a-dab0fd42811f admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:13.444878 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1296/2580] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:24:13 2022] PUT /identity/v3/projects/3e4444b1ae8340c4b07216158d2fc761/users/66a7634943ea49fc9b571b7b6acad938/roles/e0ab41d8ae22474688b5361bcc75e0f0 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 20:24:13.449702 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7b2cdf-ec64-4cdb-839a-dab0fd42811f admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:13.472745 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:24:13.498109 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:24:13.506601 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b57106d-89b8-409c-bb3c-0f9a7ceb03bf admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:13.570990 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7b2cdf-ec64-4cdb-839a-dab0fd42811f admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:13.577224 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06ab571-20b7-419b-bb60-c0d98acb1bd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:13.577899 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06ab571-20b7-419b-bb60-c0d98acb1bd5 admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:13.578181 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06ab571-20b7-419b-bb60-c0d98acb1bd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:13.578455 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06ab571-20b7-419b-bb60-c0d98acb1bd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:13.579906 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06ab571-20b7-419b-bb60-c0d98acb1bd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:13.580941 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06ab571-20b7-419b-bb60-c0d98acb1bd5 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:13.593292 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1297/2581] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:24:13 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 20:24:13.598447 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06ab571-20b7-419b-bb60-c0d98acb1bd5 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:13.610900 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06ab571-20b7-419b-bb60-c0d98acb1bd5 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:13.616047 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7c842c-2775-4619-8fc9-6472dfbb4fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:13.616691 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7c842c-2775-4619-8fc9-6472dfbb4fe7 admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:13.616969 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7c842c-2775-4619-8fc9-6472dfbb4fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:13.617239 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7c842c-2775-4619-8fc9-6472dfbb4fe7 admin admin] PATH_INFO: `/v3/users` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:13.617946 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7c842c-2775-4619-8fc9-6472dfbb4fe7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:13.618861 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822bd60e-3f89-4928-b5eb-9439f7b5f5b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:13.620048 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822bd60e-3f89-4928-b5eb-9439f7b5f5b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:13.620233 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7c842c-2775-4619-8fc9-6472dfbb4fe7 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:13.620566 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822bd60e-3f89-4928-b5eb-9439f7b5f5b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:13.621130 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822bd60e-3f89-4928-b5eb-9439f7b5f5b7 admin admin] PATH_INFO: `/v3/projects/3e4444b1ae8340c4b07216158d2fc761/users/66a7634943ea49fc9b571b7b6acad938/roles/e0ab41d8ae22474688b5361bcc75e0f0` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:13.624894 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822bd60e-3f89-4928-b5eb-9439f7b5f5b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e4444b1ae8340c4b07216158d2fc761, user_id=66a7634943ea49fc9b571b7b6acad938, role_id=e0ab41d8ae22474688b5361bcc75e0f0)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:13.630756 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822bd60e-3f89-4928-b5eb-9439f7b5f5b7 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:13.653004 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1285/2582] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:24:13 2022] POST /identity/v3/users => generated 287 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:24:13.657774 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7c842c-2775-4619-8fc9-6472dfbb4fe7 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:13.658507 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1298/2583] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:24:13 2022] PUT /identity/v3/projects/3e4444b1ae8340c4b07216158d2fc761/users/66a7634943ea49fc9b571b7b6acad938/roles/e0ab41d8ae22474688b5361bcc75e0f0 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 20:24:13.670486 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd54936-038a-446b-8da3-0bc70a7e5af6 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:13.670853 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd54936-038a-446b-8da3-0bc70a7e5af6 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:13.671281 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd54936-038a-446b-8da3-0bc70a7e5af6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:13.673953 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7c842c-2775-4619-8fc9-6472dfbb4fe7 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:13.715040 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-dbd54936-038a-446b-8da3-0bc70a7e5af6 None None] MFA Rules not processed for user `66a7634943ea49fc9b571b7b6acad938`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:24:13.717375 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-dbd54936-038a-446b-8da3-0bc70a7e5af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:13.757309 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66befdfb-442e-47ae-8f9c-95e2793a85bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:13.757980 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66befdfb-442e-47ae-8f9c-95e2793a85bb admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:13.758255 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66befdfb-442e-47ae-8f9c-95e2793a85bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:13.758530 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66befdfb-442e-47ae-8f9c-95e2793a85bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:13.759205 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66befdfb-442e-47ae-8f9c-95e2793a85bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:13.760889 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66befdfb-442e-47ae-8f9c-95e2793a85bb admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:13.773956 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1286/2584] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:24:13 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 20:24:13.778159 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66befdfb-442e-47ae-8f9c-95e2793a85bb admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:13.789901 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66befdfb-442e-47ae-8f9c-95e2793a85bb admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:13.795662 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a930ae32-392e-4fb7-a338-9ea07a4b037a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:13.796278 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a930ae32-392e-4fb7-a338-9ea07a4b037a admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:13.796565 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a930ae32-392e-4fb7-a338-9ea07a4b037a admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:13.796885 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a930ae32-392e-4fb7-a338-9ea07a4b037a admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:13.797593 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a930ae32-392e-4fb7-a338-9ea07a4b037a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:13.799397 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a930ae32-392e-4fb7-a338-9ea07a4b037a admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:13.817153 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1287/2585] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:24:13 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 20:24:13.819886 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a930ae32-392e-4fb7-a338-9ea07a4b037a admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:13.830722 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1299/2586] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:24:13 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:24:13.835456 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a930ae32-392e-4fb7-a338-9ea07a4b037a admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:13.842856 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091629ec-a4fd-493e-85bf-7906f9573a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:13.843707 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091629ec-a4fd-493e-85bf-7906f9573a33 admin admin] REQUEST_METHOD: `PUT` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:13.845596 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd54936-038a-446b-8da3-0bc70a7e5af6 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:13.846168 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091629ec-a4fd-493e-85bf-7906f9573a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:13.847066 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091629ec-a4fd-493e-85bf-7906f9573a33 admin admin] PATH_INFO: `/v3/projects/41416e1465f541b6b41f2086f750f8ef/users/8493c3f84ba34cd1800e08189f91f619/roles/4eea39e24b06431aa357053fe7907a3d` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:13.848743 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-dbd54936-038a-446b-8da3-0bc70a7e5af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:13.878305 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091629ec-a4fd-493e-85bf-7906f9573a33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41416e1465f541b6b41f2086f750f8ef, user_id=8493c3f84ba34cd1800e08189f91f619, role_id=4eea39e24b06431aa357053fe7907a3d)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:13.884474 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091629ec-a4fd-493e-85bf-7906f9573a33 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:13.905581 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1288/2587] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:24:13 2022] PUT /identity/v3/projects/41416e1465f541b6b41f2086f750f8ef/users/8493c3f84ba34cd1800e08189f91f619/roles/4eea39e24b06431aa357053fe7907a3d => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:24:13.921431 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e22ece1-78fe-44c6-a094-40a4958cab8c None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:13.921878 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e22ece1-78fe-44c6-a094-40a4958cab8c None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:13.922220 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e22ece1-78fe-44c6-a094-40a4958cab8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:13.932206 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd54936-038a-446b-8da3-0bc70a7e5af6 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:13.961012 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-5e22ece1-78fe-44c6-a094-40a4958cab8c None None] MFA Rules not processed for user `8493c3f84ba34cd1800e08189f91f619`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:24:13.962686 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-5e22ece1-78fe-44c6-a094-40a4958cab8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:14.007662 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68abdfba-4410-4dae-bd51-2ea1a0c7b950 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df1bca993bc142908cc7cf85eabd2b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:14.008300 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68abdfba-4410-4dae-bd51-2ea1a0c7b950 service cinder] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:14.008688 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68abdfba-4410-4dae-bd51-2ea1a0c7b950 service cinder] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:14.009030 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68abdfba-4410-4dae-bd51-2ea1a0c7b950 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:14.013140 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:24:14.014225 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:24:14.076533 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1289/2588] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:24:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:24:14.093913 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e22ece1-78fe-44c6-a094-40a4958cab8c None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:14.094844 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68abdfba-4410-4dae-bd51-2ea1a0c7b950 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:14.096418 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-5e22ece1-78fe-44c6-a094-40a4958cab8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:14.096822 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68abdfba-4410-4dae-bd51-2ea1a0c7b950 service cinder] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:14.109147 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1300/2589] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:24:13 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 20:24:14.237023 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e22ece1-78fe-44c6-a094-40a4958cab8c None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:14.246873 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99787d47-cf0c-49e1-a874-79abc77331f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5776cbf4f6714b2083c3ebdd6d98cc87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:14.247776 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99787d47-cf0c-49e1-a874-79abc77331f8 service nova] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:14.248130 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99787d47-cf0c-49e1-a874-79abc77331f8 service nova] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:14.248517 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99787d47-cf0c-49e1-a874-79abc77331f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:14.260442 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99787d47-cf0c-49e1-a874-79abc77331f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:14.262934 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99787d47-cf0c-49e1-a874-79abc77331f8 service nova] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:14.279571 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1290/2590] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:24:14 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:24:14.343295 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68abdfba-4410-4dae-bd51-2ea1a0c7b950 service cinder] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:14.344914 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-68abdfba-4410-4dae-bd51-2ea1a0c7b950 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:14.357254 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68abdfba-4410-4dae-bd51-2ea1a0c7b950 service cinder] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:14.361742 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee9356f-ae89-48af-aa23-ef54a768c619 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df1bca993bc142908cc7cf85eabd2b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:14.362374 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee9356f-ae89-48af-aa23-ef54a768c619 service cinder] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:14.362652 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee9356f-ae89-48af-aa23-ef54a768c619 service cinder] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:14.362932 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee9356f-ae89-48af-aa23-ef54a768c619 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:14.365406 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-aee9356f-ae89-48af-aa23-ef54a768c619 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:14.460758 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee9356f-ae89-48af-aa23-ef54a768c619 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:14.462681 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee9356f-ae89-48af-aa23-ef54a768c619 service cinder] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:14.475446 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1301/2591] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:24:14 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 20:24:17.402176 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99787d47-cf0c-49e1-a874-79abc77331f8 service nova] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:17.652360 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99787d47-cf0c-49e1-a874-79abc77331f8 service nova] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:17.658619 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277f9f39-5f50-483b-886f-98ff638f9033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:17.660231 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277f9f39-5f50-483b-886f-98ff638f9033 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:17.660702 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277f9f39-5f50-483b-886f-98ff638f9033 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:17.661210 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277f9f39-5f50-483b-886f-98ff638f9033 admin admin] PATH_INFO: `/v3/users/8493c3f84ba34cd1800e08189f91f619` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:17.663374 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277f9f39-5f50-483b-886f-98ff638f9033 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8493c3f84ba34cd1800e08189f91f619)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:17.667233 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277f9f39-5f50-483b-886f-98ff638f9033 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:17.744915 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-277f9f39-5f50-483b-886f-98ff638f9033 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8493c3f84ba34cd1800e08189f91f619'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:17.756680 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-277f9f39-5f50-483b-886f-98ff638f9033 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8493c3f84ba34cd1800e08189f91f619'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:17.758095 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-277f9f39-5f50-483b-886f-98ff638f9033 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8493c3f84ba34cd1800e08189f91f619'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:17.759293 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-277f9f39-5f50-483b-886f-98ff638f9033 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8493c3f84ba34cd1800e08189f91f619'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:17.765321 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:24:17.780430 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-277f9f39-5f50-483b-886f-98ff638f9033 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8493c3f84ba34cd1800e08189f91f619'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:17.813827 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1291/2592] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:24:17 2022] DELETE /identity/v3/users/8493c3f84ba34cd1800e08189f91f619 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:24:17.819872 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e830d5-7a66-4012-9d2d-c2b1513058ca None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:17.820713 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e830d5-7a66-4012-9d2d-c2b1513058ca None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:17.820713 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e830d5-7a66-4012-9d2d-c2b1513058ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:17.843208 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-54e830d5-7a66-4012-9d2d-c2b1513058ca None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:24:17.844507 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-54e830d5-7a66-4012-9d2d-c2b1513058ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:17.922813 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1302/2593] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:24:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:24:17.936729 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277f9f39-5f50-483b-886f-98ff638f9033 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:17.939084 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-277f9f39-5f50-483b-886f-98ff638f9033 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:18.038066 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277f9f39-5f50-483b-886f-98ff638f9033 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:18.042241 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619ce047-c2ff-4ce4-8406-0a0375be7ec5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:18.042993 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619ce047-c2ff-4ce4-8406-0a0375be7ec5 service neutron] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:18.043380 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619ce047-c2ff-4ce4-8406-0a0375be7ec5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:18.043775 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619ce047-c2ff-4ce4-8406-0a0375be7ec5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:18.057072 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619ce047-c2ff-4ce4-8406-0a0375be7ec5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:18.059500 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619ce047-c2ff-4ce4-8406-0a0375be7ec5 service neutron] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:18.078223 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1292/2594] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:24:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:24:18.494150 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e830d5-7a66-4012-9d2d-c2b1513058ca None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:18.496421 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-54e830d5-7a66-4012-9d2d-c2b1513058ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:18.511547 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e830d5-7a66-4012-9d2d-c2b1513058ca None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:18.520622 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baa73a0-e939-4a00-bcbd-07a55f419a66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:18.521750 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baa73a0-e939-4a00-bcbd-07a55f419a66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:18.522271 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baa73a0-e939-4a00-bcbd-07a55f419a66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:18.523732 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baa73a0-e939-4a00-bcbd-07a55f419a66 admin admin] PATH_INFO: `/v3/projects/41416e1465f541b6b41f2086f750f8ef` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:18.526969 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baa73a0-e939-4a00-bcbd-07a55f419a66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=41416e1465f541b6b41f2086f750f8ef)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:18.530754 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baa73a0-e939-4a00-bcbd-07a55f419a66 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:18.603000 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:24:18.622020 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-0baa73a0-e939-4a00-bcbd-07a55f419a66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41416e1465f541b6b41f2086f750f8ef', 'request_id': 'req-0baa73a0-e939-4a00-bcbd-07a55f419a66'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:18.636955 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-0baa73a0-e939-4a00-bcbd-07a55f419a66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41416e1465f541b6b41f2086f750f8ef', 'request_id': 'req-0baa73a0-e939-4a00-bcbd-07a55f419a66'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:18.649354 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-0baa73a0-e939-4a00-bcbd-07a55f419a66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41416e1465f541b6b41f2086f750f8ef', 'request_id': 'req-0baa73a0-e939-4a00-bcbd-07a55f419a66'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:18.676974 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-0baa73a0-e939-4a00-bcbd-07a55f419a66 admin admin] The token cache is being invalidate because project 41416e1465f541b6b41f2086f750f8ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81772) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:24:18.677423 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-0baa73a0-e939-4a00-bcbd-07a55f419a66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:18.678891 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-0baa73a0-e939-4a00-bcbd-07a55f419a66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:18.681601 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1303/2595] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:24:18 2022] DELETE /identity/v3/projects/41416e1465f541b6b41f2086f750f8ef => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:24:18.687646 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7beb47-c20b-4b7c-bfea-647c6184a3da None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:18.688859 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7beb47-c20b-4b7c-bfea-647c6184a3da None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:18.688859 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7beb47-c20b-4b7c-bfea-647c6184a3da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:18.715241 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-4e7beb47-c20b-4b7c-bfea-647c6184a3da None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:24:18.716631 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-4e7beb47-c20b-4b7c-bfea-647c6184a3da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:18.835957 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1293/2596] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:24:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:24:18.846228 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f54dcc-f6ba-4987-9969-817ab8c7600f None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:18.847096 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f54dcc-f6ba-4987-9969-817ab8c7600f None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:18.847096 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f54dcc-f6ba-4987-9969-817ab8c7600f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:18.869972 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-d6f54dcc-f6ba-4987-9969-817ab8c7600f None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:24:18.871233 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-d6f54dcc-f6ba-4987-9969-817ab8c7600f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:18.881032 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1304/2597] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:24:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:24:18.885635 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7beb47-c20b-4b7c-bfea-647c6184a3da None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:18.897416 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7beb47-c20b-4b7c-bfea-647c6184a3da None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:18.903247 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bab6fbe-e036-4941-93ca-4f7ba7f08818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:18.903941 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bab6fbe-e036-4941-93ca-4f7ba7f08818 admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:18.904269 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bab6fbe-e036-4941-93ca-4f7ba7f08818 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:18.904588 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bab6fbe-e036-4941-93ca-4f7ba7f08818 admin admin] PATH_INFO: `/v3/domains` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:18.905443 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bab6fbe-e036-4941-93ca-4f7ba7f08818 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:24:18.905907 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bab6fbe-e036-4941-93ca-4f7ba7f08818 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:18.907841 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bab6fbe-e036-4941-93ca-4f7ba7f08818 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:18.924452 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1294/2598] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 6 20:24:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:24:18.930402 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f54dcc-f6ba-4987-9969-817ab8c7600f None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:18.942195 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f54dcc-f6ba-4987-9969-817ab8c7600f None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:18.947681 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1279312a-45f5-4c29-8373-f5da7a4ec7ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:18.948355 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1279312a-45f5-4c29-8373-f5da7a4ec7ae admin admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:18.948727 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1279312a-45f5-4c29-8373-f5da7a4ec7ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:18.949011 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1279312a-45f5-4c29-8373-f5da7a4ec7ae admin admin] PATH_INFO: `/v3/projects` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:18.949698 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1279312a-45f5-4c29-8373-f5da7a4ec7ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:18.951681 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1279312a-45f5-4c29-8373-f5da7a4ec7ae admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:18.996170 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1305/2599] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:24:18 2022] POST /identity/v3/projects => generated 403 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:24:19.001450 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bab6fbe-e036-4941-93ca-4f7ba7f08818 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:19.014155 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bab6fbe-e036-4941-93ca-4f7ba7f08818 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:19.098640 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c73dc97-dde2-4bb4-a9b9-40f3fca2ad6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:19.099319 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c73dc97-dde2-4bb4-a9b9-40f3fca2ad6d admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:19.099848 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c73dc97-dde2-4bb4-a9b9-40f3fca2ad6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:19.100187 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c73dc97-dde2-4bb4-a9b9-40f3fca2ad6d admin admin] PATH_INFO: `/v3/users` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:19.101064 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c73dc97-dde2-4bb4-a9b9-40f3fca2ad6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:19.103293 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c73dc97-dde2-4bb4-a9b9-40f3fca2ad6d admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:19.115021 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:24:19.137698 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1295/2600] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:24:19 2022] POST /identity/v3/users => generated 302 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 20:24:19.140663 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1279312a-45f5-4c29-8373-f5da7a4ec7ae admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:19.156262 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1279312a-45f5-4c29-8373-f5da7a4ec7ae admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:19.163244 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be97d7a4-a6a1-4161-8903-7e680b123d1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:19.163929 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be97d7a4-a6a1-4161-8903-7e680b123d1a admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:19.164254 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be97d7a4-a6a1-4161-8903-7e680b123d1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:19.164574 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be97d7a4-a6a1-4161-8903-7e680b123d1a admin admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:19.165332 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be97d7a4-a6a1-4161-8903-7e680b123d1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:19.167369 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be97d7a4-a6a1-4161-8903-7e680b123d1a admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:19.180553 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1306/2601] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:24:19 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:24:19.185504 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c73dc97-dde2-4bb4-a9b9-40f3fca2ad6d admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:19.198600 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c73dc97-dde2-4bb4-a9b9-40f3fca2ad6d admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:19.206017 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed43fe9f-b3af-4259-9180-bbb238281ff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:19.206732 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed43fe9f-b3af-4259-9180-bbb238281ff0 admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:19.207145 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed43fe9f-b3af-4259-9180-bbb238281ff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:19.207434 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed43fe9f-b3af-4259-9180-bbb238281ff0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:19.208350 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed43fe9f-b3af-4259-9180-bbb238281ff0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:19.211353 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed43fe9f-b3af-4259-9180-bbb238281ff0 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:19.229025 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1296/2602] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:24:19 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:24:19.233910 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be97d7a4-a6a1-4161-8903-7e680b123d1a admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:19.245088 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be97d7a4-a6a1-4161-8903-7e680b123d1a admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:19.250400 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99186ca-d764-46fb-9882-5a6d13a31125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:19.251210 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99186ca-d764-46fb-9882-5a6d13a31125 admin admin] REQUEST_METHOD: `PUT` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:19.251591 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99186ca-d764-46fb-9882-5a6d13a31125 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:19.251968 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99186ca-d764-46fb-9882-5a6d13a31125 admin admin] PATH_INFO: `/v3/projects/3e01548ce6c04ec288cf6a4c6eec8871/users/99c9206a9c6c42d78e570b8312dcc775/roles/4eea39e24b06431aa357053fe7907a3d` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:19.272361 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99186ca-d764-46fb-9882-5a6d13a31125 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e01548ce6c04ec288cf6a4c6eec8871, user_id=99c9206a9c6c42d78e570b8312dcc775, role_id=4eea39e24b06431aa357053fe7907a3d)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:19.278467 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99186ca-d764-46fb-9882-5a6d13a31125 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:19.302428 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1307/2603] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:24:19 2022] PUT /identity/v3/projects/3e01548ce6c04ec288cf6a4c6eec8871/users/99c9206a9c6c42d78e570b8312dcc775/roles/4eea39e24b06431aa357053fe7907a3d => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 20:24:19.320749 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c2464f-fbdb-45db-955d-f26368cd56fc None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:19.321236 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c2464f-fbdb-45db-955d-f26368cd56fc None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:19.321741 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c2464f-fbdb-45db-955d-f26368cd56fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:19.357167 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-56c2464f-fbdb-45db-955d-f26368cd56fc None None] MFA Rules not processed for user `99c9206a9c6c42d78e570b8312dcc775`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:24:19.358377 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-56c2464f-fbdb-45db-955d-f26368cd56fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:19.464403 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1297/2604] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:24:19 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:24:19.470579 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99186ca-d764-46fb-9882-5a6d13a31125 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:19.573757 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99186ca-d764-46fb-9882-5a6d13a31125 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:19.579093 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4454e54-f8e4-49d0-9873-ceda9e312561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:19.579719 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4454e54-f8e4-49d0-9873-ceda9e312561 admin admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:19.580034 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4454e54-f8e4-49d0-9873-ceda9e312561 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:19.580312 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4454e54-f8e4-49d0-9873-ceda9e312561 admin admin] PATH_INFO: `/v3/projects` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:19.580994 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4454e54-f8e4-49d0-9873-ceda9e312561 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:19.583636 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4454e54-f8e4-49d0-9873-ceda9e312561 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:19.620377 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1308/2605] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:24:19 2022] POST /identity/v3/projects => generated 405 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 20:24:19.625189 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c2464f-fbdb-45db-955d-f26368cd56fc None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:19.645771 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c2464f-fbdb-45db-955d-f26368cd56fc None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:19.741108 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479b3d96-856d-489a-b453-72e196cde435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:19.742190 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479b3d96-856d-489a-b453-72e196cde435 admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:19.742407 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479b3d96-856d-489a-b453-72e196cde435 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:19.742866 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479b3d96-856d-489a-b453-72e196cde435 admin admin] PATH_INFO: `/v3/users` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:19.743660 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479b3d96-856d-489a-b453-72e196cde435 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:19.746072 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479b3d96-856d-489a-b453-72e196cde435 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:19.784142 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1298/2606] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:24:19 2022] POST /identity/v3/users => generated 309 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:24:19.786438 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4454e54-f8e4-49d0-9873-ceda9e312561 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:19.799817 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4454e54-f8e4-49d0-9873-ceda9e312561 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:19.806317 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fee79f-34c4-4af9-b903-a2560e95cc95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:19.807088 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fee79f-34c4-4af9-b903-a2560e95cc95 admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:19.807468 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fee79f-34c4-4af9-b903-a2560e95cc95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:19.807840 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fee79f-34c4-4af9-b903-a2560e95cc95 admin admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:19.808664 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fee79f-34c4-4af9-b903-a2560e95cc95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:19.811030 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fee79f-34c4-4af9-b903-a2560e95cc95 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:19.833460 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1309/2607] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:24:19 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:24:19.838340 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479b3d96-856d-489a-b453-72e196cde435 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:19.853984 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479b3d96-856d-489a-b453-72e196cde435 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:19.861019 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f50b86-4310-4292-a2b7-4e17e7093f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:19.862113 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f50b86-4310-4292-a2b7-4e17e7093f48 admin admin] REQUEST_METHOD: `PUT` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:19.863393 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f50b86-4310-4292-a2b7-4e17e7093f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:19.863891 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f50b86-4310-4292-a2b7-4e17e7093f48 admin admin] PATH_INFO: `/v3/projects/8ac02367754842879212c83f1453c6b2/users/79c3650baddd47eba1867652160bb372/roles/e0ab41d8ae22474688b5361bcc75e0f0` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:19.885853 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f50b86-4310-4292-a2b7-4e17e7093f48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ac02367754842879212c83f1453c6b2, user_id=79c3650baddd47eba1867652160bb372, role_id=e0ab41d8ae22474688b5361bcc75e0f0)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:19.888555 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f50b86-4310-4292-a2b7-4e17e7093f48 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:19.906954 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1299/2608] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:24:19 2022] PUT /identity/v3/projects/8ac02367754842879212c83f1453c6b2/users/79c3650baddd47eba1867652160bb372/roles/e0ab41d8ae22474688b5361bcc75e0f0 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:24:19.911997 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fee79f-34c4-4af9-b903-a2560e95cc95 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:19.926278 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fee79f-34c4-4af9-b903-a2560e95cc95 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:19.977096 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:24:20.013991 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a7ad5f-cf30-42f8-8bc8-e2395d6cb441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:20.014921 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a7ad5f-cf30-42f8-8bc8-e2395d6cb441 admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:20.015206 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a7ad5f-cf30-42f8-8bc8-e2395d6cb441 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:20.015481 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a7ad5f-cf30-42f8-8bc8-e2395d6cb441 admin admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:20.016185 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a7ad5f-cf30-42f8-8bc8-e2395d6cb441 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:20.017904 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a7ad5f-cf30-42f8-8bc8-e2395d6cb441 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:20.036829 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1310/2609] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:24:19 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:24:20.042576 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f50b86-4310-4292-a2b7-4e17e7093f48 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:20.063276 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f50b86-4310-4292-a2b7-4e17e7093f48 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:20.070844 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e48c0d-42be-4001-ab41-7c9271833dc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:20.072022 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e48c0d-42be-4001-ab41-7c9271833dc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:20.072546 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e48c0d-42be-4001-ab41-7c9271833dc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:20.073060 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e48c0d-42be-4001-ab41-7c9271833dc4 admin admin] PATH_INFO: `/v3/projects/8ac02367754842879212c83f1453c6b2/users/79c3650baddd47eba1867652160bb372/roles/e0ab41d8ae22474688b5361bcc75e0f0` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:20.076959 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e48c0d-42be-4001-ab41-7c9271833dc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ac02367754842879212c83f1453c6b2, user_id=79c3650baddd47eba1867652160bb372, role_id=e0ab41d8ae22474688b5361bcc75e0f0)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:20.082987 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e48c0d-42be-4001-ab41-7c9271833dc4 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:20.101012 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1300/2610] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:24:20 2022] PUT /identity/v3/projects/8ac02367754842879212c83f1453c6b2/users/79c3650baddd47eba1867652160bb372/roles/e0ab41d8ae22474688b5361bcc75e0f0 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:24:20.124698 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fce61ec-75fe-4146-b9aa-231c8fe6c321 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:20.125155 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fce61ec-75fe-4146-b9aa-231c8fe6c321 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:20.125655 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fce61ec-75fe-4146-b9aa-231c8fe6c321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:20.174801 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-6fce61ec-75fe-4146-b9aa-231c8fe6c321 None None] MFA Rules not processed for user `79c3650baddd47eba1867652160bb372`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:24:20.177285 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-6fce61ec-75fe-4146-b9aa-231c8fe6c321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:20.290160 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1311/2611] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:24:20 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:24:20.309997 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e48c0d-42be-4001-ab41-7c9271833dc4 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:20.312781 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-01e48c0d-42be-4001-ab41-7c9271833dc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:20.350118 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:24:20.392474 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e48c0d-42be-4001-ab41-7c9271833dc4 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:20.419730 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465281df-5d5c-4f31-bcd0-2747c1419055 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df1bca993bc142908cc7cf85eabd2b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:20.422013 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465281df-5d5c-4f31-bcd0-2747c1419055 service cinder] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:20.422849 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465281df-5d5c-4f31-bcd0-2747c1419055 service cinder] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:20.423344 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465281df-5d5c-4f31-bcd0-2747c1419055 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:20.441230 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465281df-5d5c-4f31-bcd0-2747c1419055 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:20.442923 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465281df-5d5c-4f31-bcd0-2747c1419055 service cinder] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:20.458924 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1301/2612] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:24:20 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:24:20.812936 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fce61ec-75fe-4146-b9aa-231c8fe6c321 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:20.901330 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fce61ec-75fe-4146-b9aa-231c8fe6c321 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:20.910237 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0f9874-8da6-4d71-8041-4c47e6607497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:20.911377 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0f9874-8da6-4d71-8041-4c47e6607497 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:20.911874 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0f9874-8da6-4d71-8041-4c47e6607497 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:20.912351 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0f9874-8da6-4d71-8041-4c47e6607497 admin admin] PATH_INFO: `/v3/users/99c9206a9c6c42d78e570b8312dcc775` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:20.914564 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0f9874-8da6-4d71-8041-4c47e6607497 admin admin] RBAC: Authorizing `identity:delete_user(user_id=99c9206a9c6c42d78e570b8312dcc775)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:20.918299 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0f9874-8da6-4d71-8041-4c47e6607497 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:20.990012 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-4a0f9874-8da6-4d71-8041-4c47e6607497 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99c9206a9c6c42d78e570b8312dcc775'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:20.990589 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-4a0f9874-8da6-4d71-8041-4c47e6607497 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99c9206a9c6c42d78e570b8312dcc775'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:20.999547 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:24:21.008277 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-4a0f9874-8da6-4d71-8041-4c47e6607497 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99c9206a9c6c42d78e570b8312dcc775'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:21.009503 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-4a0f9874-8da6-4d71-8041-4c47e6607497 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99c9206a9c6c42d78e570b8312dcc775'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:21.018518 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-4a0f9874-8da6-4d71-8041-4c47e6607497 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99c9206a9c6c42d78e570b8312dcc775'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:21.035511 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1312/2613] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:24:20 2022] DELETE /identity/v3/users/99c9206a9c6c42d78e570b8312dcc775 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:24:21.041652 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9779547-296e-4dcb-9848-ea9df7a58dbf None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:21.042193 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9779547-296e-4dcb-9848-ea9df7a58dbf None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:21.042379 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9779547-296e-4dcb-9848-ea9df7a58dbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:21.068706 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-b9779547-296e-4dcb-9848-ea9df7a58dbf None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:24:21.069953 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-b9779547-296e-4dcb-9848-ea9df7a58dbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:21.157110 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1302/2614] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:24:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:24:21.169644 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0f9874-8da6-4d71-8041-4c47e6607497 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:21.171463 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-4a0f9874-8da6-4d71-8041-4c47e6607497 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:21.268282 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0f9874-8da6-4d71-8041-4c47e6607497 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:21.274477 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92812517-bf07-45e6-85d2-491e0a4fa626 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:21.275121 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92812517-bf07-45e6-85d2-491e0a4fa626 service neutron] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:21.275519 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92812517-bf07-45e6-85d2-491e0a4fa626 service neutron] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:21.275930 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92812517-bf07-45e6-85d2-491e0a4fa626 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:21.290022 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92812517-bf07-45e6-85d2-491e0a4fa626 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:21.292094 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92812517-bf07-45e6-85d2-491e0a4fa626 service neutron] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:21.306984 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1313/2615] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:24:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:24:21.692350 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9779547-296e-4dcb-9848-ea9df7a58dbf None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:21.694221 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-b9779547-296e-4dcb-9848-ea9df7a58dbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:21.713139 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9779547-296e-4dcb-9848-ea9df7a58dbf None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:21.719226 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6bf741-e0c6-4b51-aa72-2f37295d726d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:21.720286 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6bf741-e0c6-4b51-aa72-2f37295d726d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:21.720757 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6bf741-e0c6-4b51-aa72-2f37295d726d admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:21.721232 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6bf741-e0c6-4b51-aa72-2f37295d726d admin admin] PATH_INFO: `/v3/projects/3e01548ce6c04ec288cf6a4c6eec8871` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:21.723154 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6bf741-e0c6-4b51-aa72-2f37295d726d admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e01548ce6c04ec288cf6a4c6eec8871)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:21.727010 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6bf741-e0c6-4b51-aa72-2f37295d726d admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:21.799214 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-3c6bf741-e0c6-4b51-aa72-2f37295d726d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e01548ce6c04ec288cf6a4c6eec8871', 'request_id': 'req-3c6bf741-e0c6-4b51-aa72-2f37295d726d'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:21.809277 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-3c6bf741-e0c6-4b51-aa72-2f37295d726d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e01548ce6c04ec288cf6a4c6eec8871', 'request_id': 'req-3c6bf741-e0c6-4b51-aa72-2f37295d726d'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:21.823948 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-3c6bf741-e0c6-4b51-aa72-2f37295d726d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e01548ce6c04ec288cf6a4c6eec8871', 'request_id': 'req-3c6bf741-e0c6-4b51-aa72-2f37295d726d'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:21.849146 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-3c6bf741-e0c6-4b51-aa72-2f37295d726d admin admin] The token cache is being invalidate because project 3e01548ce6c04ec288cf6a4c6eec8871 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81771) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:24:21.849654 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-3c6bf741-e0c6-4b51-aa72-2f37295d726d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:21.850797 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-3c6bf741-e0c6-4b51-aa72-2f37295d726d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:21.853367 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1303/2616] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:24:21 2022] DELETE /identity/v3/projects/3e01548ce6c04ec288cf6a4c6eec8871 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:24:21.858826 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92812517-bf07-45e6-85d2-491e0a4fa626 service neutron] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:21.876806 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92812517-bf07-45e6-85d2-491e0a4fa626 service neutron] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:21.975796 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a336b4f-6f4b-4eea-aa38-b6cec9542109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:21.976751 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a336b4f-6f4b-4eea-aa38-b6cec9542109 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:21.977017 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a336b4f-6f4b-4eea-aa38-b6cec9542109 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:21.977497 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a336b4f-6f4b-4eea-aa38-b6cec9542109 admin admin] PATH_INFO: `/v3/users/79c3650baddd47eba1867652160bb372` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:21.979294 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a336b4f-6f4b-4eea-aa38-b6cec9542109 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79c3650baddd47eba1867652160bb372)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:21.981855 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a336b4f-6f4b-4eea-aa38-b6cec9542109 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:22.052570 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-3a336b4f-6f4b-4eea-aa38-b6cec9542109 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79c3650baddd47eba1867652160bb372'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:22.053387 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-3a336b4f-6f4b-4eea-aa38-b6cec9542109 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79c3650baddd47eba1867652160bb372'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:22.071268 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-3a336b4f-6f4b-4eea-aa38-b6cec9542109 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79c3650baddd47eba1867652160bb372'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:22.072219 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-3a336b4f-6f4b-4eea-aa38-b6cec9542109 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79c3650baddd47eba1867652160bb372'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:22.083516 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-3a336b4f-6f4b-4eea-aa38-b6cec9542109 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79c3650baddd47eba1867652160bb372'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:22.092351 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:24:22.109853 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1314/2617] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:24:21 2022] DELETE /identity/v3/users/79c3650baddd47eba1867652160bb372 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:24:22.681386 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6bf741-e0c6-4b51-aa72-2f37295d726d admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:22.686557 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-3c6bf741-e0c6-4b51-aa72-2f37295d726d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:22.704463 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a336b4f-6f4b-4eea-aa38-b6cec9542109 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:22.711015 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-3a336b4f-6f4b-4eea-aa38-b6cec9542109 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:22.725369 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6bf741-e0c6-4b51-aa72-2f37295d726d admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:22.818962 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:24:22.839360 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f018fd0-c0f4-40fa-a47e-3aa71952f585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:22.841480 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f018fd0-c0f4-40fa-a47e-3aa71952f585 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:22.841480 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f018fd0-c0f4-40fa-a47e-3aa71952f585 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:22.841940 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f018fd0-c0f4-40fa-a47e-3aa71952f585 admin admin] PATH_INFO: `/v3/projects/8ac02367754842879212c83f1453c6b2` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:22.844239 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f018fd0-c0f4-40fa-a47e-3aa71952f585 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ac02367754842879212c83f1453c6b2)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:22.845880 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a336b4f-6f4b-4eea-aa38-b6cec9542109 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:22.848125 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f018fd0-c0f4-40fa-a47e-3aa71952f585 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:22.856448 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65baef1-e20a-4a79-875c-eee97ccf158e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:22.857512 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65baef1-e20a-4a79-875c-eee97ccf158e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:22.858025 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65baef1-e20a-4a79-875c-eee97ccf158e admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:22.858803 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65baef1-e20a-4a79-875c-eee97ccf158e admin admin] PATH_INFO: `/v3/users/154e87a9077f4738995c62beaddcafe2` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:22.861912 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65baef1-e20a-4a79-875c-eee97ccf158e admin admin] RBAC: Authorizing `identity:delete_user(user_id=154e87a9077f4738995c62beaddcafe2)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:22.865962 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65baef1-e20a-4a79-875c-eee97ccf158e admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:22.935311 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-7f018fd0-c0f4-40fa-a47e-3aa71952f585 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ac02367754842879212c83f1453c6b2', 'request_id': 'req-7f018fd0-c0f4-40fa-a47e-3aa71952f585'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:22.948535 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-7f018fd0-c0f4-40fa-a47e-3aa71952f585 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ac02367754842879212c83f1453c6b2', 'request_id': 'req-7f018fd0-c0f4-40fa-a47e-3aa71952f585'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:22.956377 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-f65baef1-e20a-4a79-875c-eee97ccf158e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '154e87a9077f4738995c62beaddcafe2'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:22.957802 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-f65baef1-e20a-4a79-875c-eee97ccf158e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '154e87a9077f4738995c62beaddcafe2'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:22.963877 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-7f018fd0-c0f4-40fa-a47e-3aa71952f585 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ac02367754842879212c83f1453c6b2', 'request_id': 'req-7f018fd0-c0f4-40fa-a47e-3aa71952f585'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:22.982377 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-7f018fd0-c0f4-40fa-a47e-3aa71952f585 admin admin] The token cache is being invalidate because project 8ac02367754842879212c83f1453c6b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81771) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:24:22.982894 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-7f018fd0-c0f4-40fa-a47e-3aa71952f585 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:22.983817 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-7f018fd0-c0f4-40fa-a47e-3aa71952f585 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:22.987675 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1304/2618] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:24:22 2022] DELETE /identity/v3/projects/8ac02367754842879212c83f1453c6b2 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:24:22.996165 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-f65baef1-e20a-4a79-875c-eee97ccf158e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '154e87a9077f4738995c62beaddcafe2'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:22.997657 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80894ef6-4a92-415f-bda1-3a5605a5df6a None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:22.998344 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80894ef6-4a92-415f-bda1-3a5605a5df6a None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:22.998663 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-f65baef1-e20a-4a79-875c-eee97ccf158e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '154e87a9077f4738995c62beaddcafe2'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:22.998833 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80894ef6-4a92-415f-bda1-3a5605a5df6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:23.016015 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-f65baef1-e20a-4a79-875c-eee97ccf158e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '154e87a9077f4738995c62beaddcafe2'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:23.038059 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-80894ef6-4a92-415f-bda1-3a5605a5df6a None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:24:23.040632 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-80894ef6-4a92-415f-bda1-3a5605a5df6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:23.044561 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1315/2619] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:24:22 2022] DELETE /identity/v3/users/154e87a9077f4738995c62beaddcafe2 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:24:23.052050 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540e56e1-eb91-4bf0-bea4-6205909e7f9a None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:23.052523 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540e56e1-eb91-4bf0-bea4-6205909e7f9a None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:23.052986 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540e56e1-eb91-4bf0-bea4-6205909e7f9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:23.088783 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-540e56e1-eb91-4bf0-bea4-6205909e7f9a None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:24:23.090902 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-540e56e1-eb91-4bf0-bea4-6205909e7f9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:23.176018 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1305/2620] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:24:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:24:23.185577 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe5149c-4224-4198-9cd0-cbc0d8e28bc6 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:23.186075 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe5149c-4224-4198-9cd0-cbc0d8e28bc6 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:23.186441 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe5149c-4224-4198-9cd0-cbc0d8e28bc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:23.212799 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-bbe5149c-4224-4198-9cd0-cbc0d8e28bc6 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:24:23.215445 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-bbe5149c-4224-4198-9cd0-cbc0d8e28bc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:23.223519 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1316/2621] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:24:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:24:23.233299 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1306/2622] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:24:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:24:23.239105 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540e56e1-eb91-4bf0-bea4-6205909e7f9a None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:23.241050 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe5149c-4224-4198-9cd0-cbc0d8e28bc6 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:23.241560 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-540e56e1-eb91-4bf0-bea4-6205909e7f9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:23.258849 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe5149c-4224-4198-9cd0-cbc0d8e28bc6 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:23.267377 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cb5d10-d13a-4283-900f-54055f9b527a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:23.268418 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cb5d10-d13a-4283-900f-54055f9b527a admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:23.269430 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cb5d10-d13a-4283-900f-54055f9b527a admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:23.269848 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cb5d10-d13a-4283-900f-54055f9b527a admin admin] PATH_INFO: `/v3/domains` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:23.270993 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cb5d10-d13a-4283-900f-54055f9b527a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:24:23.271512 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cb5d10-d13a-4283-900f-54055f9b527a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:23.274315 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cb5d10-d13a-4283-900f-54055f9b527a admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:23.287779 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:24:23.296357 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1307/2623] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 6 20:24:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:24:23.303528 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cb5d10-d13a-4283-900f-54055f9b527a admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:23.324333 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cb5d10-d13a-4283-900f-54055f9b527a admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:23.333569 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953578b9-42d2-4fc6-8a75-b028509b1493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:23.334672 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953578b9-42d2-4fc6-8a75-b028509b1493 admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:23.335151 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953578b9-42d2-4fc6-8a75-b028509b1493 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:23.335599 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953578b9-42d2-4fc6-8a75-b028509b1493 admin admin] PATH_INFO: `/v3/projects` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:23.336718 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953578b9-42d2-4fc6-8a75-b028509b1493 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:23.340465 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953578b9-42d2-4fc6-8a75-b028509b1493 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:23.374482 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540e56e1-eb91-4bf0-bea4-6205909e7f9a None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:23.383202 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f616e110-ade0-465b-9a9b-d92b74119a09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:23.385571 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f616e110-ade0-465b-9a9b-d92b74119a09 service neutron] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:23.386100 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f616e110-ade0-465b-9a9b-d92b74119a09 service neutron] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:23.386982 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f616e110-ade0-465b-9a9b-d92b74119a09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:23.402587 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1308/2624] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:24:23 2022] POST /identity/v3/projects => generated 365 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 20:24:23.405535 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953578b9-42d2-4fc6-8a75-b028509b1493 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:23.407931 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f616e110-ade0-465b-9a9b-d92b74119a09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:23.410996 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f616e110-ade0-465b-9a9b-d92b74119a09 service neutron] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:23.433543 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1317/2625] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:24:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:24:23.491412 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:24:23.532968 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953578b9-42d2-4fc6-8a75-b028509b1493 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:23.543298 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e11115-bca5-44c9-ac95-4051f41cebee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:23.544511 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e11115-bca5-44c9-ac95-4051f41cebee admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:23.545131 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e11115-bca5-44c9-ac95-4051f41cebee admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:23.545933 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e11115-bca5-44c9-ac95-4051f41cebee admin admin] PATH_INFO: `/v3/users` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:23.547730 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e11115-bca5-44c9-ac95-4051f41cebee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:23.552179 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e11115-bca5-44c9-ac95-4051f41cebee admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:23.599293 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1309/2626] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:24:23 2022] POST /identity/v3/users => generated 283 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:24:23.605864 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f616e110-ade0-465b-9a9b-d92b74119a09 service neutron] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:23.623199 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f616e110-ade0-465b-9a9b-d92b74119a09 service neutron] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:23.630300 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e399cb-6ea9-4ad0-8caa-f3123db581dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:23.631388 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e399cb-6ea9-4ad0-8caa-f3123db581dc admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:23.631822 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e399cb-6ea9-4ad0-8caa-f3123db581dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:23.632235 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e399cb-6ea9-4ad0-8caa-f3123db581dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:23.633370 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e399cb-6ea9-4ad0-8caa-f3123db581dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:23.635322 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e399cb-6ea9-4ad0-8caa-f3123db581dc admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:23.654168 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1318/2627] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:24:23 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:24:23.661651 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e11115-bca5-44c9-ac95-4051f41cebee admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:23.678845 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e11115-bca5-44c9-ac95-4051f41cebee admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:23.686077 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0738095f-a6c5-4f96-a975-39b69eba6e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:23.687143 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0738095f-a6c5-4f96-a975-39b69eba6e64 admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:23.687641 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0738095f-a6c5-4f96-a975-39b69eba6e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:23.689153 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0738095f-a6c5-4f96-a975-39b69eba6e64 admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:23.690749 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0738095f-a6c5-4f96-a975-39b69eba6e64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:23.695272 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0738095f-a6c5-4f96-a975-39b69eba6e64 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:23.716007 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1310/2628] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:24:23 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:24:23.720861 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e399cb-6ea9-4ad0-8caa-f3123db581dc admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:23.738711 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e399cb-6ea9-4ad0-8caa-f3123db581dc admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:23.745172 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef351961-164d-4256-a84f-a18a533f7a2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:23.746251 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef351961-164d-4256-a84f-a18a533f7a2b admin admin] REQUEST_METHOD: `PUT` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:23.746709 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef351961-164d-4256-a84f-a18a533f7a2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:23.747089 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef351961-164d-4256-a84f-a18a533f7a2b admin admin] PATH_INFO: `/v3/projects/fa440c46ffde444db340b84dbd388fee/users/605bf01f3f114452ba27caa31be11923/roles/4eea39e24b06431aa357053fe7907a3d` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:23.768281 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef351961-164d-4256-a84f-a18a533f7a2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa440c46ffde444db340b84dbd388fee, user_id=605bf01f3f114452ba27caa31be11923, role_id=4eea39e24b06431aa357053fe7907a3d)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:23.771368 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef351961-164d-4256-a84f-a18a533f7a2b admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:23.798783 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1319/2629] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:24:23 2022] PUT /identity/v3/projects/fa440c46ffde444db340b84dbd388fee/users/605bf01f3f114452ba27caa31be11923/roles/4eea39e24b06431aa357053fe7907a3d => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:24:23.804554 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5374404-14b1-4c38-b030-2f973d125cc2 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:23.804909 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5374404-14b1-4c38-b030-2f973d125cc2 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:23.805191 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5374404-14b1-4c38-b030-2f973d125cc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:23.847819 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-b5374404-14b1-4c38-b030-2f973d125cc2 None None] MFA Rules not processed for user `605bf01f3f114452ba27caa31be11923`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:24:23.849394 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-b5374404-14b1-4c38-b030-2f973d125cc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:23.997265 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1311/2630] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:24:23 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:24:24.004873 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef351961-164d-4256-a84f-a18a533f7a2b admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:24.155878 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef351961-164d-4256-a84f-a18a533f7a2b admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:24.165792 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14dc65e-7b91-4e43-8161-9ff2971e0dee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:24.167229 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14dc65e-7b91-4e43-8161-9ff2971e0dee admin admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:24.167769 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14dc65e-7b91-4e43-8161-9ff2971e0dee admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:24.168303 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14dc65e-7b91-4e43-8161-9ff2971e0dee admin admin] PATH_INFO: `/v3/projects` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:24.169537 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14dc65e-7b91-4e43-8161-9ff2971e0dee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:24.173607 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14dc65e-7b91-4e43-8161-9ff2971e0dee admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:24.194170 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5374404-14b1-4c38-b030-2f973d125cc2 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:24.197574 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-b5374404-14b1-4c38-b030-2f973d125cc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:24.221740 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5374404-14b1-4c38-b030-2f973d125cc2 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:24.228928 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a61fc7-c835-4611-af8b-d13ced17cccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:24.229726 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a61fc7-c835-4611-af8b-d13ced17cccc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:24.230062 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a61fc7-c835-4611-af8b-d13ced17cccc admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:24.230515 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a61fc7-c835-4611-af8b-d13ced17cccc admin admin] PATH_INFO: `/v3/projects/840ba46c70e3484c830bdc4b5da5020a` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:24.232224 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a61fc7-c835-4611-af8b-d13ced17cccc admin admin] RBAC: Authorizing `identity:delete_project(project_id=840ba46c70e3484c830bdc4b5da5020a)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:24.234620 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a61fc7-c835-4611-af8b-d13ced17cccc admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:24.239080 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1320/2631] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:24:24 2022] POST /identity/v3/projects => generated 365 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 20:24:24.241820 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14dc65e-7b91-4e43-8161-9ff2971e0dee admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:24.312930 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-34a61fc7-c835-4611-af8b-d13ced17cccc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '840ba46c70e3484c830bdc4b5da5020a', 'request_id': 'req-34a61fc7-c835-4611-af8b-d13ced17cccc'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:24.322364 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-34a61fc7-c835-4611-af8b-d13ced17cccc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '840ba46c70e3484c830bdc4b5da5020a', 'request_id': 'req-34a61fc7-c835-4611-af8b-d13ced17cccc'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:24.336051 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-34a61fc7-c835-4611-af8b-d13ced17cccc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '840ba46c70e3484c830bdc4b5da5020a', 'request_id': 'req-34a61fc7-c835-4611-af8b-d13ced17cccc'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:24.354486 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-34a61fc7-c835-4611-af8b-d13ced17cccc admin admin] The token cache is being invalidate because project 840ba46c70e3484c830bdc4b5da5020a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81771) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:24:24.355450 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-34a61fc7-c835-4611-af8b-d13ced17cccc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:24.356316 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-34a61fc7-c835-4611-af8b-d13ced17cccc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:24.358548 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1312/2632] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:24:24 2022] DELETE /identity/v3/projects/840ba46c70e3484c830bdc4b5da5020a => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 20:24:24.363392 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a61fc7-c835-4611-af8b-d13ced17cccc admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:24.366195 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-34a61fc7-c835-4611-af8b-d13ced17cccc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:24.396248 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14dc65e-7b91-4e43-8161-9ff2971e0dee admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:24.423251 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:24:24.432879 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:24:24.516937 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a61fc7-c835-4611-af8b-d13ced17cccc admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:24.522862 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de515011-4cfc-4295-b489-32bf180d1285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:24.523605 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de515011-4cfc-4295-b489-32bf180d1285 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:24.523977 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de515011-4cfc-4295-b489-32bf180d1285 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:24.524430 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de515011-4cfc-4295-b489-32bf180d1285 admin admin] PATH_INFO: `/v3/users/66a7634943ea49fc9b571b7b6acad938` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:24.526079 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de515011-4cfc-4295-b489-32bf180d1285 admin admin] RBAC: Authorizing `identity:delete_user(user_id=66a7634943ea49fc9b571b7b6acad938)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:24.528688 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de515011-4cfc-4295-b489-32bf180d1285 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:24.542452 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9373d1ce-ff4b-4fd3-8d77-0c7302514520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:24.544361 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9373d1ce-ff4b-4fd3-8d77-0c7302514520 admin admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:24.544361 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9373d1ce-ff4b-4fd3-8d77-0c7302514520 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:24.544954 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9373d1ce-ff4b-4fd3-8d77-0c7302514520 admin admin] PATH_INFO: `/v3/users` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:24.548922 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9373d1ce-ff4b-4fd3-8d77-0c7302514520 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:24.554041 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9373d1ce-ff4b-4fd3-8d77-0c7302514520 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:24.629738 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1321/2633] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:24:24 2022] POST /identity/v3/users => generated 289 bytes in 389 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:24:24.644789 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9373d1ce-ff4b-4fd3-8d77-0c7302514520 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:24.647423 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-de515011-4cfc-4295-b489-32bf180d1285 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66a7634943ea49fc9b571b7b6acad938'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:24.650961 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-9373d1ce-ff4b-4fd3-8d77-0c7302514520 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:24.674645 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-de515011-4cfc-4295-b489-32bf180d1285 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66a7634943ea49fc9b571b7b6acad938'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:24.676457 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-de515011-4cfc-4295-b489-32bf180d1285 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66a7634943ea49fc9b571b7b6acad938'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:24.677824 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-de515011-4cfc-4295-b489-32bf180d1285 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66a7634943ea49fc9b571b7b6acad938'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:24.694201 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9373d1ce-ff4b-4fd3-8d77-0c7302514520 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:24.698529 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-de515011-4cfc-4295-b489-32bf180d1285 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66a7634943ea49fc9b571b7b6acad938'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:24.704453 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01674ede-5acc-46e7-ac60-745bd75300d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:24.705515 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01674ede-5acc-46e7-ac60-745bd75300d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:24.707081 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01674ede-5acc-46e7-ac60-745bd75300d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:24.707081 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01674ede-5acc-46e7-ac60-745bd75300d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:24.708336 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01674ede-5acc-46e7-ac60-745bd75300d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:24.711603 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01674ede-5acc-46e7-ac60-745bd75300d5 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:24.739094 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1322/2634] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:24:24 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:24:24.741469 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01674ede-5acc-46e7-ac60-745bd75300d5 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:24.744680 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-01674ede-5acc-46e7-ac60-745bd75300d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:24.762840 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1313/2635] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:24:24 2022] DELETE /identity/v3/users/66a7634943ea49fc9b571b7b6acad938 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:24:24.784964 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01674ede-5acc-46e7-ac60-745bd75300d5 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:24.963864 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691ad9c1-0c11-49c0-af80-3bdfd69156af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:24.964600 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691ad9c1-0c11-49c0-af80-3bdfd69156af admin admin] REQUEST_METHOD: `PUT` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:24.965044 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691ad9c1-0c11-49c0-af80-3bdfd69156af admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:24.965340 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691ad9c1-0c11-49c0-af80-3bdfd69156af admin admin] PATH_INFO: `/v3/projects/b27a2ab6ffe4417c9de5cf375f9979f6/users/d8d36d59c2a14347b482b3baed2d3866/roles/e0ab41d8ae22474688b5361bcc75e0f0` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:25.013325 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691ad9c1-0c11-49c0-af80-3bdfd69156af admin admin] RBAC: Authorizing `identity:create_grant(project_id=b27a2ab6ffe4417c9de5cf375f9979f6, user_id=d8d36d59c2a14347b482b3baed2d3866, role_id=e0ab41d8ae22474688b5361bcc75e0f0)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:25.036676 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691ad9c1-0c11-49c0-af80-3bdfd69156af admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:25.082564 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1323/2636] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:24:24 2022] PUT /identity/v3/projects/b27a2ab6ffe4417c9de5cf375f9979f6/users/d8d36d59c2a14347b482b3baed2d3866/roles/e0ab41d8ae22474688b5361bcc75e0f0 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 20:24:25.089164 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de515011-4cfc-4295-b489-32bf180d1285 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:25.115054 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de515011-4cfc-4295-b489-32bf180d1285 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:25.257470 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dcacdd-53e7-4c8f-b9f5-7fe2a23b94a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:25.258598 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dcacdd-53e7-4c8f-b9f5-7fe2a23b94a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:25.259024 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dcacdd-53e7-4c8f-b9f5-7fe2a23b94a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:25.261728 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dcacdd-53e7-4c8f-b9f5-7fe2a23b94a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:25.263156 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dcacdd-53e7-4c8f-b9f5-7fe2a23b94a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:25.267253 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dcacdd-53e7-4c8f-b9f5-7fe2a23b94a3 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:25.288430 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1314/2637] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:24:25 2022] GET /identity/v3/roles => generated 1420 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:24:25.294608 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691ad9c1-0c11-49c0-af80-3bdfd69156af admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:25.315196 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691ad9c1-0c11-49c0-af80-3bdfd69156af admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:25.327250 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41274881-1722-4e2a-b9a8-540ddf46ca0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:25.328689 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41274881-1722-4e2a-b9a8-540ddf46ca0b admin admin] REQUEST_METHOD: `PUT` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:25.329190 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41274881-1722-4e2a-b9a8-540ddf46ca0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:25.329935 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41274881-1722-4e2a-b9a8-540ddf46ca0b admin admin] PATH_INFO: `/v3/projects/b27a2ab6ffe4417c9de5cf375f9979f6/users/d8d36d59c2a14347b482b3baed2d3866/roles/e0ab41d8ae22474688b5361bcc75e0f0` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:25.335122 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41274881-1722-4e2a-b9a8-540ddf46ca0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b27a2ab6ffe4417c9de5cf375f9979f6, user_id=d8d36d59c2a14347b482b3baed2d3866, role_id=e0ab41d8ae22474688b5361bcc75e0f0)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:25.341034 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41274881-1722-4e2a-b9a8-540ddf46ca0b admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:25.377091 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1324/2638] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:24:25 2022] PUT /identity/v3/projects/b27a2ab6ffe4417c9de5cf375f9979f6/users/d8d36d59c2a14347b482b3baed2d3866/roles/e0ab41d8ae22474688b5361bcc75e0f0 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:24:25.400047 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ef62b2-0b59-45d4-a1c7-57fd4dc2d56c None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:25.404075 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ef62b2-0b59-45d4-a1c7-57fd4dc2d56c None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:25.404646 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ef62b2-0b59-45d4-a1c7-57fd4dc2d56c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:25.430966 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41274881-1722-4e2a-b9a8-540ddf46ca0b admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:25.434882 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-41274881-1722-4e2a-b9a8-540ddf46ca0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:25.472765 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-c5ef62b2-0b59-45d4-a1c7-57fd4dc2d56c None None] MFA Rules not processed for user `d8d36d59c2a14347b482b3baed2d3866`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:24:25.474661 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-c5ef62b2-0b59-45d4-a1c7-57fd4dc2d56c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:25.561746 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:24:25.624093 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1315/2639] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:24:25 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:24:25.642689 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41274881-1722-4e2a-b9a8-540ddf46ca0b admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:25.646637 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ef62b2-0b59-45d4-a1c7-57fd4dc2d56c None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:25.648552 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-c5ef62b2-0b59-45d4-a1c7-57fd4dc2d56c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:25.651410 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99dd015-0d6c-425c-8958-d64160eaf621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:25.651410 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99dd015-0d6c-425c-8958-d64160eaf621 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:25.651410 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99dd015-0d6c-425c-8958-d64160eaf621 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:25.651410 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99dd015-0d6c-425c-8958-d64160eaf621 admin admin] PATH_INFO: `/v3/projects/3e4444b1ae8340c4b07216158d2fc761` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:25.652993 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99dd015-0d6c-425c-8958-d64160eaf621 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e4444b1ae8340c4b07216158d2fc761)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:25.655893 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99dd015-0d6c-425c-8958-d64160eaf621 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:25.665388 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:24:25.735175 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-c99dd015-0d6c-425c-8958-d64160eaf621 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e4444b1ae8340c4b07216158d2fc761', 'request_id': 'req-c99dd015-0d6c-425c-8958-d64160eaf621'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:25.747976 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-c99dd015-0d6c-425c-8958-d64160eaf621 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e4444b1ae8340c4b07216158d2fc761', 'request_id': 'req-c99dd015-0d6c-425c-8958-d64160eaf621'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:25.750707 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ef62b2-0b59-45d4-a1c7-57fd4dc2d56c None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:25.764080 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-c99dd015-0d6c-425c-8958-d64160eaf621 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e4444b1ae8340c4b07216158d2fc761', 'request_id': 'req-c99dd015-0d6c-425c-8958-d64160eaf621'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:25.783151 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-c99dd015-0d6c-425c-8958-d64160eaf621 admin admin] The token cache is being invalidate because project 3e4444b1ae8340c4b07216158d2fc761 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81772) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:24:25.785146 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-c99dd015-0d6c-425c-8958-d64160eaf621 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:25.785146 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-c99dd015-0d6c-425c-8958-d64160eaf621 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:25.789752 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1325/2640] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:24:25 2022] DELETE /identity/v3/projects/3e4444b1ae8340c4b07216158d2fc761 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 20:24:25.792321 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223cb12c-a74b-4bf0-8d9e-1aba70bfa14c None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:25.792755 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223cb12c-a74b-4bf0-8d9e-1aba70bfa14c None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:25.793111 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223cb12c-a74b-4bf0-8d9e-1aba70bfa14c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:25.818660 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-223cb12c-a74b-4bf0-8d9e-1aba70bfa14c None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:24:25.820105 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-223cb12c-a74b-4bf0-8d9e-1aba70bfa14c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:25.832022 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7bd445-f064-4cb8-9943-64936396886d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df1bca993bc142908cc7cf85eabd2b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:25.833201 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7bd445-f064-4cb8-9943-64936396886d service cinder] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:25.833757 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7bd445-f064-4cb8-9943-64936396886d service cinder] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:25.834461 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7bd445-f064-4cb8-9943-64936396886d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:25.838544 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-9d7bd445-f064-4cb8-9943-64936396886d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:25.930902 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1326/2641] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:24:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:24:25.942304 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ddf5b4-aaa6-4a33-b0a6-99d19c840953 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:25.942943 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ddf5b4-aaa6-4a33-b0a6-99d19c840953 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:25.943539 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ddf5b4-aaa6-4a33-b0a6-99d19c840953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:25.950122 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7bd445-f064-4cb8-9943-64936396886d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:25.954657 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7bd445-f064-4cb8-9943-64936396886d service cinder] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:25.974120 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1316/2642] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:24:25 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:24:25.985151 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-d0ddf5b4-aaa6-4a33-b0a6-99d19c840953 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:24:25.987443 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-d0ddf5b4-aaa6-4a33-b0a6-99d19c840953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:26.009688 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1327/2643] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:24:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:24:26.011417 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7bd445-f064-4cb8-9943-64936396886d service cinder] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:26.035870 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7bd445-f064-4cb8-9943-64936396886d service cinder] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:26.041335 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b103d82-820d-41d1-b9d9-dc9ae4bdab19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:26.042024 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b103d82-820d-41d1-b9d9-dc9ae4bdab19 admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:26.042353 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b103d82-820d-41d1-b9d9-dc9ae4bdab19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:26.043086 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b103d82-820d-41d1-b9d9-dc9ae4bdab19 admin admin] PATH_INFO: `/v3/domains` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:26.044019 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b103d82-820d-41d1-b9d9-dc9ae4bdab19 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:24:26.044326 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b103d82-820d-41d1-b9d9-dc9ae4bdab19 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:26.046181 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b103d82-820d-41d1-b9d9-dc9ae4bdab19 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:26.065522 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1317/2644] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 6 20:24:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 20:24:26.070869 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ddf5b4-aaa6-4a33-b0a6-99d19c840953 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:26.086302 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ddf5b4-aaa6-4a33-b0a6-99d19c840953 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:26.096731 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32e52e9-28d9-46f6-bbfd-977214fcc230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:26.097663 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32e52e9-28d9-46f6-bbfd-977214fcc230 admin admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:26.097992 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32e52e9-28d9-46f6-bbfd-977214fcc230 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:26.098491 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32e52e9-28d9-46f6-bbfd-977214fcc230 admin admin] PATH_INFO: `/v3/projects` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:26.099413 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32e52e9-28d9-46f6-bbfd-977214fcc230 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:26.109956 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32e52e9-28d9-46f6-bbfd-977214fcc230 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:26.124572 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b103d82-820d-41d1-b9d9-dc9ae4bdab19 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:26.127386 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-5b103d82-820d-41d1-b9d9-dc9ae4bdab19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:26.155486 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b103d82-820d-41d1-b9d9-dc9ae4bdab19 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:26.163871 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fdb69a-fd49-47bf-8779-c0a6c4f77a1b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df1bca993bc142908cc7cf85eabd2b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:26.164974 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fdb69a-fd49-47bf-8779-c0a6c4f77a1b service cinder] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:26.165540 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fdb69a-fd49-47bf-8779-c0a6c4f77a1b service cinder] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:26.166872 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fdb69a-fd49-47bf-8779-c0a6c4f77a1b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:26.169849 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-50fdb69a-fd49-47bf-8779-c0a6c4f77a1b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:26.171538 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1328/2645] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:24:26 2022] POST /identity/v3/projects => generated 371 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 20:24:26.173335 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32e52e9-28d9-46f6-bbfd-977214fcc230 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:26.209742 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:24:26.292262 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fdb69a-fd49-47bf-8779-c0a6c4f77a1b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:26.294282 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fdb69a-fd49-47bf-8779-c0a6c4f77a1b service cinder] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:26.307675 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1318/2646] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:24:26 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 20:24:26.314047 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32e52e9-28d9-46f6-bbfd-977214fcc230 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:26.322593 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79030464-6cb4-4a68-b1ee-e5ea1890af35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:26.323368 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79030464-6cb4-4a68-b1ee-e5ea1890af35 admin admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:26.324079 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79030464-6cb4-4a68-b1ee-e5ea1890af35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:26.324591 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79030464-6cb4-4a68-b1ee-e5ea1890af35 admin admin] PATH_INFO: `/v3/users` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:26.325299 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79030464-6cb4-4a68-b1ee-e5ea1890af35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:26.327614 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79030464-6cb4-4a68-b1ee-e5ea1890af35 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:26.342734 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:24:26.375624 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fdb69a-fd49-47bf-8779-c0a6c4f77a1b service cinder] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:26.384533 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1329/2647] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:24:26 2022] POST /identity/v3/users => generated 286 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 20:24:26.400269 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fdb69a-fd49-47bf-8779-c0a6c4f77a1b service cinder] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:26.409977 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad1d7c3-8476-4f49-8f54-b550de6c7e34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:26.410876 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad1d7c3-8476-4f49-8f54-b550de6c7e34 admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:26.411208 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad1d7c3-8476-4f49-8f54-b550de6c7e34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:26.411735 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad1d7c3-8476-4f49-8f54-b550de6c7e34 admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:26.412639 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad1d7c3-8476-4f49-8f54-b550de6c7e34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:26.415499 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad1d7c3-8476-4f49-8f54-b550de6c7e34 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:26.437678 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1319/2648] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:24:26 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:24:26.444594 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79030464-6cb4-4a68-b1ee-e5ea1890af35 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:26.460238 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79030464-6cb4-4a68-b1ee-e5ea1890af35 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:26.472821 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91b63c3-494a-4e9f-a342-cd84f06a2429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:26.474451 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91b63c3-494a-4e9f-a342-cd84f06a2429 admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:26.477209 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91b63c3-494a-4e9f-a342-cd84f06a2429 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:26.477782 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91b63c3-494a-4e9f-a342-cd84f06a2429 admin admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:26.485973 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91b63c3-494a-4e9f-a342-cd84f06a2429 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:26.489708 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91b63c3-494a-4e9f-a342-cd84f06a2429 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:26.513937 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1330/2649] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:24:26 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 20:24:26.519167 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad1d7c3-8476-4f49-8f54-b550de6c7e34 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:26.545019 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad1d7c3-8476-4f49-8f54-b550de6c7e34 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:26.554958 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80060f2e-1dc1-4e93-b5cd-399c34a200ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:26.556101 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80060f2e-1dc1-4e93-b5cd-399c34a200ee admin admin] REQUEST_METHOD: `PUT` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:26.556584 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80060f2e-1dc1-4e93-b5cd-399c34a200ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:26.557069 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80060f2e-1dc1-4e93-b5cd-399c34a200ee admin admin] PATH_INFO: `/v3/projects/c05cd2d805834013a01d51e4f0528fea/users/ea970be1bdbb43ff9918b5a19924d835/roles/4eea39e24b06431aa357053fe7907a3d` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:26.593223 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80060f2e-1dc1-4e93-b5cd-399c34a200ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=c05cd2d805834013a01d51e4f0528fea, user_id=ea970be1bdbb43ff9918b5a19924d835, role_id=4eea39e24b06431aa357053fe7907a3d)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:26.598580 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80060f2e-1dc1-4e93-b5cd-399c34a200ee admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:26.640617 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1320/2650] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:24:26 2022] PUT /identity/v3/projects/c05cd2d805834013a01d51e4f0528fea/users/ea970be1bdbb43ff9918b5a19924d835/roles/4eea39e24b06431aa357053fe7907a3d => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:24:26.653792 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e983efed-891e-492a-bbe5-4e408b0dcf41 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:26.655468 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e983efed-891e-492a-bbe5-4e408b0dcf41 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:26.655982 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e983efed-891e-492a-bbe5-4e408b0dcf41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:26.710427 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-e983efed-891e-492a-bbe5-4e408b0dcf41 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:24:26.712836 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-e983efed-891e-492a-bbe5-4e408b0dcf41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:26.861029 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1331/2651] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:24:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:24:26.880978 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80060f2e-1dc1-4e93-b5cd-399c34a200ee admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:26.884916 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-80060f2e-1dc1-4e93-b5cd-399c34a200ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:27.007352 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80060f2e-1dc1-4e93-b5cd-399c34a200ee admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:27.016998 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9647fa56-b0b7-4041-b872-cd2c9afedd8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:27.018244 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9647fa56-b0b7-4041-b872-cd2c9afedd8e service neutron] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:27.019323 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9647fa56-b0b7-4041-b872-cd2c9afedd8e service neutron] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:27.019725 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9647fa56-b0b7-4041-b872-cd2c9afedd8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:27.041218 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9647fa56-b0b7-4041-b872-cd2c9afedd8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:27.044024 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9647fa56-b0b7-4041-b872-cd2c9afedd8e service neutron] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:27.062639 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1321/2652] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:24:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:24:28.228268 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326952a6-dea1-46dc-b478-3761256e984b None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:28.229141 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326952a6-dea1-46dc-b478-3761256e984b None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:28.229141 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326952a6-dea1-46dc-b478-3761256e984b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:28.282806 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-326952a6-dea1-46dc-b478-3761256e984b None None] MFA Rules not processed for user `ea970be1bdbb43ff9918b5a19924d835`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:24:28.286033 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-326952a6-dea1-46dc-b478-3761256e984b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:28.438806 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1332/2653] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:24:28 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:24:28.444773 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9647fa56-b0b7-4041-b872-cd2c9afedd8e service neutron] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:28.459572 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9647fa56-b0b7-4041-b872-cd2c9afedd8e service neutron] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:28.466610 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ea656f-7377-450b-b193-307c1cba0127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:28.467467 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ea656f-7377-450b-b193-307c1cba0127 admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:28.467848 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ea656f-7377-450b-b193-307c1cba0127 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:28.468244 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ea656f-7377-450b-b193-307c1cba0127 admin admin] PATH_INFO: `/v3/projects` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:28.469120 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ea656f-7377-450b-b193-307c1cba0127 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:28.471675 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ea656f-7377-450b-b193-307c1cba0127 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:28.514444 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1322/2654] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:24:28 2022] POST /identity/v3/projects => generated 373 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:24:28.520315 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326952a6-dea1-46dc-b478-3761256e984b None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:28.538153 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326952a6-dea1-46dc-b478-3761256e984b None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:28.637364 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bedb2e7-6426-44b3-a0d5-fe8e479f3b40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:28.638617 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bedb2e7-6426-44b3-a0d5-fe8e479f3b40 admin admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:28.639076 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bedb2e7-6426-44b3-a0d5-fe8e479f3b40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:28.639615 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bedb2e7-6426-44b3-a0d5-fe8e479f3b40 admin admin] PATH_INFO: `/v3/users` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:28.640792 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bedb2e7-6426-44b3-a0d5-fe8e479f3b40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:28.644603 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bedb2e7-6426-44b3-a0d5-fe8e479f3b40 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:28.683596 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1333/2655] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:24:28 2022] POST /identity/v3/users => generated 293 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:24:28.688663 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ea656f-7377-450b-b193-307c1cba0127 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:28.704258 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ea656f-7377-450b-b193-307c1cba0127 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:28.712910 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab164da-e73d-4c4d-b6a9-7bed1c902745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:28.717561 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab164da-e73d-4c4d-b6a9-7bed1c902745 admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:28.717561 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab164da-e73d-4c4d-b6a9-7bed1c902745 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:28.717561 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab164da-e73d-4c4d-b6a9-7bed1c902745 admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:28.723823 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab164da-e73d-4c4d-b6a9-7bed1c902745 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:28.725275 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab164da-e73d-4c4d-b6a9-7bed1c902745 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:28.740330 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1323/2656] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:24:28 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:24:28.745986 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bedb2e7-6426-44b3-a0d5-fe8e479f3b40 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:28.759537 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:24:28.767562 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bedb2e7-6426-44b3-a0d5-fe8e479f3b40 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:28.792457 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c0bbab-b27e-4b52-861f-48c010dd35af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:28.793604 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c0bbab-b27e-4b52-861f-48c010dd35af admin admin] REQUEST_METHOD: `PUT` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:28.794125 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c0bbab-b27e-4b52-861f-48c010dd35af admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:28.794583 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c0bbab-b27e-4b52-861f-48c010dd35af admin admin] PATH_INFO: `/v3/projects/99a4bd4b2653468698cd20c18b864920/users/b3f39e4c4a2944aa9957bdee39d2d26e/roles/e0ab41d8ae22474688b5361bcc75e0f0` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:28.819651 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c0bbab-b27e-4b52-861f-48c010dd35af admin admin] RBAC: Authorizing `identity:create_grant(project_id=99a4bd4b2653468698cd20c18b864920, user_id=b3f39e4c4a2944aa9957bdee39d2d26e, role_id=e0ab41d8ae22474688b5361bcc75e0f0)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:28.822568 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c0bbab-b27e-4b52-861f-48c010dd35af admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:28.838932 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1334/2657] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:24:28 2022] PUT /identity/v3/projects/99a4bd4b2653468698cd20c18b864920/users/b3f39e4c4a2944aa9957bdee39d2d26e/roles/e0ab41d8ae22474688b5361bcc75e0f0 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 20:24:28.843409 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab164da-e73d-4c4d-b6a9-7bed1c902745 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:28.855671 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab164da-e73d-4c4d-b6a9-7bed1c902745 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:28.930039 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:24:28.947678 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372dd3c9-6fa8-4d7e-95c6-c1af0fd43a63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:28.948747 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372dd3c9-6fa8-4d7e-95c6-c1af0fd43a63 admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:28.949237 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372dd3c9-6fa8-4d7e-95c6-c1af0fd43a63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:28.949782 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372dd3c9-6fa8-4d7e-95c6-c1af0fd43a63 admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:28.950971 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372dd3c9-6fa8-4d7e-95c6-c1af0fd43a63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:28.954000 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372dd3c9-6fa8-4d7e-95c6-c1af0fd43a63 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:28.973693 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1324/2658] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:24:28 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 20:24:28.978769 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c0bbab-b27e-4b52-861f-48c010dd35af admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:28.996615 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c0bbab-b27e-4b52-861f-48c010dd35af admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:29.005328 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1a562d-7dac-47b4-b3bd-81fa20885024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:29.006494 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1a562d-7dac-47b4-b3bd-81fa20885024 admin admin] REQUEST_METHOD: `PUT` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:29.006984 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1a562d-7dac-47b4-b3bd-81fa20885024 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:29.007470 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1a562d-7dac-47b4-b3bd-81fa20885024 admin admin] PATH_INFO: `/v3/projects/99a4bd4b2653468698cd20c18b864920/users/b3f39e4c4a2944aa9957bdee39d2d26e/roles/e0ab41d8ae22474688b5361bcc75e0f0` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:29.011171 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1a562d-7dac-47b4-b3bd-81fa20885024 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99a4bd4b2653468698cd20c18b864920, user_id=b3f39e4c4a2944aa9957bdee39d2d26e, role_id=e0ab41d8ae22474688b5361bcc75e0f0)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:29.015547 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1a562d-7dac-47b4-b3bd-81fa20885024 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:29.036105 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1335/2659] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:24:28 2022] PUT /identity/v3/projects/99a4bd4b2653468698cd20c18b864920/users/b3f39e4c4a2944aa9957bdee39d2d26e/roles/e0ab41d8ae22474688b5361bcc75e0f0 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:24:29.971518 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5374bff5-9058-4b26-a569-98064f37d852 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:29.972519 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5374bff5-9058-4b26-a569-98064f37d852 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:29.972519 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5374bff5-9058-4b26-a569-98064f37d852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:30.047093 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-5374bff5-9058-4b26-a569-98064f37d852 None None] MFA Rules not processed for user `b3f39e4c4a2944aa9957bdee39d2d26e`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:24:30.050807 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-5374bff5-9058-4b26-a569-98064f37d852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:30.205171 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1325/2660] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:24:29 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:24:30.221023 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1a562d-7dac-47b4-b3bd-81fa20885024 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:30.224698 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-8a1a562d-7dac-47b4-b3bd-81fa20885024 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:30.231781 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5374bff5-9058-4b26-a569-98064f37d852 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:30.359666 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5374bff5-9058-4b26-a569-98064f37d852 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:30.363221 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1a562d-7dac-47b4-b3bd-81fa20885024 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:30.367529 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251f9dad-6625-41af-9022-1708c3e5a8cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:30.368534 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251f9dad-6625-41af-9022-1708c3e5a8cf service neutron] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:30.368752 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f20f262-221f-4737-930e-688d6134001e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df1bca993bc142908cc7cf85eabd2b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:30.369084 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251f9dad-6625-41af-9022-1708c3e5a8cf service neutron] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:30.369254 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f20f262-221f-4737-930e-688d6134001e service cinder] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:30.369444 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f20f262-221f-4737-930e-688d6134001e service cinder] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:30.369693 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251f9dad-6625-41af-9022-1708c3e5a8cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:30.369949 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f20f262-221f-4737-930e-688d6134001e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:30.372329 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-251f9dad-6625-41af-9022-1708c3e5a8cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:30.461469 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f20f262-221f-4737-930e-688d6134001e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:30.464534 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f20f262-221f-4737-930e-688d6134001e service cinder] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:30.484414 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1336/2661] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:24:30 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:24:30.522788 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251f9dad-6625-41af-9022-1708c3e5a8cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:30.526221 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251f9dad-6625-41af-9022-1708c3e5a8cf service neutron] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:30.549193 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1326/2662] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:24:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 20:24:30.675657 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f20f262-221f-4737-930e-688d6134001e service cinder] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:30.679145 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-2f20f262-221f-4737-930e-688d6134001e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:30.707292 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f20f262-221f-4737-930e-688d6134001e service cinder] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:30.714775 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76901a70-e0c3-47d4-8854-14fc256c5d52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:30.716386 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76901a70-e0c3-47d4-8854-14fc256c5d52 service neutron] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:30.716985 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76901a70-e0c3-47d4-8854-14fc256c5d52 service neutron] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:30.717863 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76901a70-e0c3-47d4-8854-14fc256c5d52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:30.721238 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-76901a70-e0c3-47d4-8854-14fc256c5d52 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:30.886013 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76901a70-e0c3-47d4-8854-14fc256c5d52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:30.891228 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76901a70-e0c3-47d4-8854-14fc256c5d52 service neutron] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:30.934188 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1337/2663] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:24:30 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:24:32.193115 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251f9dad-6625-41af-9022-1708c3e5a8cf service neutron] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:32.220272 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251f9dad-6625-41af-9022-1708c3e5a8cf service neutron] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:32.230041 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399f5a0b-c8c5-499d-9030-a1c5290a586f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df1bca993bc142908cc7cf85eabd2b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:32.232063 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399f5a0b-c8c5-499d-9030-a1c5290a586f service cinder] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:32.232772 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399f5a0b-c8c5-499d-9030-a1c5290a586f service cinder] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:32.233256 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399f5a0b-c8c5-499d-9030-a1c5290a586f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:32.254178 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399f5a0b-c8c5-499d-9030-a1c5290a586f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:32.257244 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399f5a0b-c8c5-499d-9030-a1c5290a586f service cinder] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:32.286985 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1327/2664] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:24:32 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:24:33.051940 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76901a70-e0c3-47d4-8854-14fc256c5d52 service neutron] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:33.053875 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-76901a70-e0c3-47d4-8854-14fc256c5d52 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:33.071747 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76901a70-e0c3-47d4-8854-14fc256c5d52 service neutron] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:33.080623 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e267f5f-8a7a-404e-9ca0-6b3797c5f51b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:33.082527 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e267f5f-8a7a-404e-9ca0-6b3797c5f51b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:33.082527 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e267f5f-8a7a-404e-9ca0-6b3797c5f51b admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:33.082957 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e267f5f-8a7a-404e-9ca0-6b3797c5f51b admin admin] PATH_INFO: `/v3/users/605bf01f3f114452ba27caa31be11923` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:33.084117 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e267f5f-8a7a-404e-9ca0-6b3797c5f51b admin admin] RBAC: Authorizing `identity:delete_user(user_id=605bf01f3f114452ba27caa31be11923)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:33.086260 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e267f5f-8a7a-404e-9ca0-6b3797c5f51b admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:33.106324 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:24:33.202430 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-7e267f5f-8a7a-404e-9ca0-6b3797c5f51b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '605bf01f3f114452ba27caa31be11923'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:33.203538 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-7e267f5f-8a7a-404e-9ca0-6b3797c5f51b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '605bf01f3f114452ba27caa31be11923'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:33.228843 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-7e267f5f-8a7a-404e-9ca0-6b3797c5f51b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '605bf01f3f114452ba27caa31be11923'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:33.229667 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-7e267f5f-8a7a-404e-9ca0-6b3797c5f51b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '605bf01f3f114452ba27caa31be11923'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:33.243991 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-7e267f5f-8a7a-404e-9ca0-6b3797c5f51b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '605bf01f3f114452ba27caa31be11923'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:33.270783 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1338/2665] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:24:33 2022] DELETE /identity/v3/users/605bf01f3f114452ba27caa31be11923 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:24:33.277272 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b225fa83-28e1-4d23-8416-14df011f80e2 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:33.277931 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b225fa83-28e1-4d23-8416-14df011f80e2 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:33.278647 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b225fa83-28e1-4d23-8416-14df011f80e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:33.315936 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-b225fa83-28e1-4d23-8416-14df011f80e2 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:24:33.318482 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-b225fa83-28e1-4d23-8416-14df011f80e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:33.339199 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:24:33.423912 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1328/2666] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:24:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:24:33.441092 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e267f5f-8a7a-404e-9ca0-6b3797c5f51b admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:33.442937 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-7e267f5f-8a7a-404e-9ca0-6b3797c5f51b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:33.570991 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e267f5f-8a7a-404e-9ca0-6b3797c5f51b admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:33.578718 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65865a47-b5b6-4997-b6a3-b2a7645fda7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:33.579817 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65865a47-b5b6-4997-b6a3-b2a7645fda7c service neutron] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:33.580539 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65865a47-b5b6-4997-b6a3-b2a7645fda7c service neutron] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:33.581653 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65865a47-b5b6-4997-b6a3-b2a7645fda7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:33.609074 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65865a47-b5b6-4997-b6a3-b2a7645fda7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:33.611803 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65865a47-b5b6-4997-b6a3-b2a7645fda7c service neutron] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:33.643075 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1339/2667] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:24:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:24:34.675431 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b225fa83-28e1-4d23-8416-14df011f80e2 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:34.680986 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-b225fa83-28e1-4d23-8416-14df011f80e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:34.712672 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b225fa83-28e1-4d23-8416-14df011f80e2 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:34.723499 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52fb1b4-290f-4b1a-b945-6e1161701f24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:34.724396 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52fb1b4-290f-4b1a-b945-6e1161701f24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:34.724788 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52fb1b4-290f-4b1a-b945-6e1161701f24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:34.725241 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52fb1b4-290f-4b1a-b945-6e1161701f24 admin admin] PATH_INFO: `/v3/projects/fa440c46ffde444db340b84dbd388fee` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:34.727031 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52fb1b4-290f-4b1a-b945-6e1161701f24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa440c46ffde444db340b84dbd388fee)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:34.729199 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52fb1b4-290f-4b1a-b945-6e1161701f24 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:34.847617 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-f52fb1b4-290f-4b1a-b945-6e1161701f24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa440c46ffde444db340b84dbd388fee', 'request_id': 'req-f52fb1b4-290f-4b1a-b945-6e1161701f24'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:34.867723 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-f52fb1b4-290f-4b1a-b945-6e1161701f24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa440c46ffde444db340b84dbd388fee', 'request_id': 'req-f52fb1b4-290f-4b1a-b945-6e1161701f24'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:34.896890 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-f52fb1b4-290f-4b1a-b945-6e1161701f24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa440c46ffde444db340b84dbd388fee', 'request_id': 'req-f52fb1b4-290f-4b1a-b945-6e1161701f24'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:34.929211 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-f52fb1b4-290f-4b1a-b945-6e1161701f24 admin admin] The token cache is being invalidate because project fa440c46ffde444db340b84dbd388fee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81771) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:24:34.929768 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-f52fb1b4-290f-4b1a-b945-6e1161701f24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:34.930656 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-f52fb1b4-290f-4b1a-b945-6e1161701f24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:34.933313 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1329/2668] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:24:34 2022] DELETE /identity/v3/projects/fa440c46ffde444db340b84dbd388fee => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:24:34.939916 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65865a47-b5b6-4997-b6a3-b2a7645fda7c service neutron] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:34.959158 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65865a47-b5b6-4997-b6a3-b2a7645fda7c service neutron] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:35.074637 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5f58c1-d526-4b5a-86bf-e3d82da8f905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:35.075540 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5f58c1-d526-4b5a-86bf-e3d82da8f905 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:35.076057 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5f58c1-d526-4b5a-86bf-e3d82da8f905 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:35.076445 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5f58c1-d526-4b5a-86bf-e3d82da8f905 admin admin] PATH_INFO: `/v3/users/d8d36d59c2a14347b482b3baed2d3866` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:35.078661 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5f58c1-d526-4b5a-86bf-e3d82da8f905 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8d36d59c2a14347b482b3baed2d3866)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:35.082407 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5f58c1-d526-4b5a-86bf-e3d82da8f905 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:35.162018 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:24:35.205007 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-ef5f58c1-d526-4b5a-86bf-e3d82da8f905 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8d36d59c2a14347b482b3baed2d3866'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:35.208294 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-ef5f58c1-d526-4b5a-86bf-e3d82da8f905 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8d36d59c2a14347b482b3baed2d3866'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:35.231773 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-ef5f58c1-d526-4b5a-86bf-e3d82da8f905 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8d36d59c2a14347b482b3baed2d3866'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:35.232839 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-ef5f58c1-d526-4b5a-86bf-e3d82da8f905 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8d36d59c2a14347b482b3baed2d3866'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:35.245338 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-ef5f58c1-d526-4b5a-86bf-e3d82da8f905 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8d36d59c2a14347b482b3baed2d3866'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:35.274091 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1340/2669] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:24:34 2022] DELETE /identity/v3/users/d8d36d59c2a14347b482b3baed2d3866 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 20:24:36.416910 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52fb1b4-290f-4b1a-b945-6e1161701f24 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:36.418890 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-f52fb1b4-290f-4b1a-b945-6e1161701f24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:36.438797 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52fb1b4-290f-4b1a-b945-6e1161701f24 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:36.548017 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cfc4d7-12b1-4d81-ac0d-bf7810e4cedc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:36.548744 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cfc4d7-12b1-4d81-ac0d-bf7810e4cedc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:36.549396 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cfc4d7-12b1-4d81-ac0d-bf7810e4cedc admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:36.550344 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cfc4d7-12b1-4d81-ac0d-bf7810e4cedc admin admin] PATH_INFO: `/v3/projects/b27a2ab6ffe4417c9de5cf375f9979f6` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:36.554615 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cfc4d7-12b1-4d81-ac0d-bf7810e4cedc admin admin] RBAC: Authorizing `identity:delete_project(project_id=b27a2ab6ffe4417c9de5cf375f9979f6)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:36.559368 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cfc4d7-12b1-4d81-ac0d-bf7810e4cedc admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:36.657409 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:24:36.661684 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-55cfc4d7-12b1-4d81-ac0d-bf7810e4cedc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b27a2ab6ffe4417c9de5cf375f9979f6', 'request_id': 'req-55cfc4d7-12b1-4d81-ac0d-bf7810e4cedc'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:36.672178 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-55cfc4d7-12b1-4d81-ac0d-bf7810e4cedc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b27a2ab6ffe4417c9de5cf375f9979f6', 'request_id': 'req-55cfc4d7-12b1-4d81-ac0d-bf7810e4cedc'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:36.688120 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-55cfc4d7-12b1-4d81-ac0d-bf7810e4cedc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b27a2ab6ffe4417c9de5cf375f9979f6', 'request_id': 'req-55cfc4d7-12b1-4d81-ac0d-bf7810e4cedc'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:36.711714 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-55cfc4d7-12b1-4d81-ac0d-bf7810e4cedc admin admin] The token cache is being invalidate because project b27a2ab6ffe4417c9de5cf375f9979f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81771) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:24:36.712073 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-55cfc4d7-12b1-4d81-ac0d-bf7810e4cedc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:36.713210 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-55cfc4d7-12b1-4d81-ac0d-bf7810e4cedc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:36.715913 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1330/2670] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:24:36 2022] DELETE /identity/v3/projects/b27a2ab6ffe4417c9de5cf375f9979f6 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 20:24:36.724817 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb71edf-9288-42c9-96fb-22f26bc4b7a2 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:36.730441 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb71edf-9288-42c9-96fb-22f26bc4b7a2 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:36.730931 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb71edf-9288-42c9-96fb-22f26bc4b7a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:36.786490 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-3fb71edf-9288-42c9-96fb-22f26bc4b7a2 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:24:36.787866 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-3fb71edf-9288-42c9-96fb-22f26bc4b7a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:37.015496 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1341/2671] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:24:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:24:37.017490 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f350e1e6-b1b0-4345-bf68-f96f20eb1129 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:37.018031 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f350e1e6-b1b0-4345-bf68-f96f20eb1129 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:37.018647 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f350e1e6-b1b0-4345-bf68-f96f20eb1129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:37.085392 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-f350e1e6-b1b0-4345-bf68-f96f20eb1129 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:24:37.087928 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-f350e1e6-b1b0-4345-bf68-f96f20eb1129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:37.147763 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb71edf-9288-42c9-96fb-22f26bc4b7a2 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:37.155595 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1331/2672] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:24:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:24:37.167065 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb71edf-9288-42c9-96fb-22f26bc4b7a2 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:37.201486 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074128f4-24d4-4c00-9019-747d406c2067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:37.202597 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074128f4-24d4-4c00-9019-747d406c2067 admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:37.203088 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074128f4-24d4-4c00-9019-747d406c2067 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:37.203592 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074128f4-24d4-4c00-9019-747d406c2067 admin admin] PATH_INFO: `/v3/domains` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:37.207265 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074128f4-24d4-4c00-9019-747d406c2067 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:24:37.207844 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074128f4-24d4-4c00-9019-747d406c2067 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:37.217353 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074128f4-24d4-4c00-9019-747d406c2067 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:37.261964 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1342/2673] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 6 20:24:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:24:37.264712 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f350e1e6-b1b0-4345-bf68-f96f20eb1129 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:37.283484 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f350e1e6-b1b0-4345-bf68-f96f20eb1129 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:37.290355 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569385b3-3b61-42d4-a237-82ebf878b09d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:37.291210 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569385b3-3b61-42d4-a237-82ebf878b09d admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:37.291589 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569385b3-3b61-42d4-a237-82ebf878b09d admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:37.292262 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569385b3-3b61-42d4-a237-82ebf878b09d admin admin] PATH_INFO: `/v3/projects` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:37.292794 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569385b3-3b61-42d4-a237-82ebf878b09d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:37.295091 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569385b3-3b61-42d4-a237-82ebf878b09d admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:37.347154 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1332/2674] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:24:37 2022] POST /identity/v3/projects => generated 375 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:24:37.352149 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074128f4-24d4-4c00-9019-747d406c2067 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:37.398697 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074128f4-24d4-4c00-9019-747d406c2067 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:37.669461 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f075ae02-71ad-48bb-8e08-a2cb4f8b9920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:37.673729 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f075ae02-71ad-48bb-8e08-a2cb4f8b9920 admin admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:37.677666 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f075ae02-71ad-48bb-8e08-a2cb4f8b9920 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:37.678290 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f075ae02-71ad-48bb-8e08-a2cb4f8b9920 admin admin] PATH_INFO: `/v3/users` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:37.679786 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f075ae02-71ad-48bb-8e08-a2cb4f8b9920 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:37.686691 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f075ae02-71ad-48bb-8e08-a2cb4f8b9920 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:37.767387 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1343/2675] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:24:37 2022] POST /identity/v3/users => generated 288 bytes in 416 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:24:37.776950 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569385b3-3b61-42d4-a237-82ebf878b09d admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:37.808726 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569385b3-3b61-42d4-a237-82ebf878b09d admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:37.829564 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38644c65-34ec-4daa-a209-d28ca5d39c33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:37.831594 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38644c65-34ec-4daa-a209-d28ca5d39c33 admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:37.832411 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38644c65-34ec-4daa-a209-d28ca5d39c33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:37.833216 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38644c65-34ec-4daa-a209-d28ca5d39c33 admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:37.835335 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38644c65-34ec-4daa-a209-d28ca5d39c33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:37.838823 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38644c65-34ec-4daa-a209-d28ca5d39c33 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:37.863346 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1333/2676] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:24:37 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 20:24:37.869302 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f075ae02-71ad-48bb-8e08-a2cb4f8b9920 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:37.885728 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f075ae02-71ad-48bb-8e08-a2cb4f8b9920 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:37.895013 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c6cb27-85ba-4319-bc47-1196309e916d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:37.895864 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c6cb27-85ba-4319-bc47-1196309e916d admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:37.896322 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c6cb27-85ba-4319-bc47-1196309e916d admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:37.896756 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c6cb27-85ba-4319-bc47-1196309e916d admin admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:37.897682 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c6cb27-85ba-4319-bc47-1196309e916d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:37.899733 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c6cb27-85ba-4319-bc47-1196309e916d admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:37.919504 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1344/2677] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:24:37 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:24:37.921236 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38644c65-34ec-4daa-a209-d28ca5d39c33 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:37.949259 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38644c65-34ec-4daa-a209-d28ca5d39c33 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:37.962008 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7fc777-50ba-4cc5-86d8-498feed18259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:37.963385 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7fc777-50ba-4cc5-86d8-498feed18259 admin admin] REQUEST_METHOD: `PUT` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:37.964166 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7fc777-50ba-4cc5-86d8-498feed18259 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:37.965379 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7fc777-50ba-4cc5-86d8-498feed18259 admin admin] PATH_INFO: `/v3/projects/7c54bded329541a8955eeda44c2006b3/users/4dd41c8e6382492d8f77c3799b215ba1/roles/4eea39e24b06431aa357053fe7907a3d` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:37.999033 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7fc777-50ba-4cc5-86d8-498feed18259 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c54bded329541a8955eeda44c2006b3, user_id=4dd41c8e6382492d8f77c3799b215ba1, role_id=4eea39e24b06431aa357053fe7907a3d)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:38.003886 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7fc777-50ba-4cc5-86d8-498feed18259 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:38.033762 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1334/2678] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:24:37 2022] PUT /identity/v3/projects/7c54bded329541a8955eeda44c2006b3/users/4dd41c8e6382492d8f77c3799b215ba1/roles/4eea39e24b06431aa357053fe7907a3d => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:24:38.053301 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f11b70-7997-43ae-898f-2023c2d4ddb7 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:38.054145 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f11b70-7997-43ae-898f-2023c2d4ddb7 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:38.054646 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f11b70-7997-43ae-898f-2023c2d4ddb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:38.063432 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7fc777-50ba-4cc5-86d8-498feed18259 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:38.070876 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-dc7fc777-50ba-4cc5-86d8-498feed18259 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:38.117033 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-41f11b70-7997-43ae-898f-2023c2d4ddb7 None None] MFA Rules not processed for user `4dd41c8e6382492d8f77c3799b215ba1`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:24:38.118492 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-41f11b70-7997-43ae-898f-2023c2d4ddb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:38.136826 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:24:38.224385 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7fc777-50ba-4cc5-86d8-498feed18259 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:38.234091 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f994e1f-4c71-4693-b764-7bd6c61a94d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:38.235236 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f994e1f-4c71-4693-b764-7bd6c61a94d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:38.235736 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f994e1f-4c71-4693-b764-7bd6c61a94d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:38.236348 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f994e1f-4c71-4693-b764-7bd6c61a94d4 admin admin] PATH_INFO: `/v3/users/7be957f2c15c4f6f949b3f284a30624d` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:38.239130 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f994e1f-4c71-4693-b764-7bd6c61a94d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7be957f2c15c4f6f949b3f284a30624d)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:38.242843 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f994e1f-4c71-4693-b764-7bd6c61a94d4 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:38.245956 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1345/2679] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:24:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:24:38.263946 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f11b70-7997-43ae-898f-2023c2d4ddb7 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:38.268179 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-41f11b70-7997-43ae-898f-2023c2d4ddb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:38.330266 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-2f994e1f-4c71-4693-b764-7bd6c61a94d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7be957f2c15c4f6f949b3f284a30624d'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:38.341465 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-2f994e1f-4c71-4693-b764-7bd6c61a94d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7be957f2c15c4f6f949b3f284a30624d'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:38.342472 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-2f994e1f-4c71-4693-b764-7bd6c61a94d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7be957f2c15c4f6f949b3f284a30624d'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:38.343119 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-2f994e1f-4c71-4693-b764-7bd6c61a94d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7be957f2c15c4f6f949b3f284a30624d'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:38.352362 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-2f994e1f-4c71-4693-b764-7bd6c61a94d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7be957f2c15c4f6f949b3f284a30624d'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:38.356315 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f11b70-7997-43ae-898f-2023c2d4ddb7 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:38.358269 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=84,keystone:DELETE=12,keystone:INSERT=4 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:24:38.363015 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e49d200-5e82-4af9-9e8b-22fda097b420 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07af18f35ffc42a295278d5349e06381', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:38.364022 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e49d200-5e82-4af9-9e8b-22fda097b420 service glance] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:38.364316 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e49d200-5e82-4af9-9e8b-22fda097b420 service glance] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:38.364597 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e49d200-5e82-4af9-9e8b-22fda097b420 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:38.367554 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-9e49d200-5e82-4af9-9e8b-22fda097b420 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:38.380716 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1335/2680] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:24:38 2022] DELETE /identity/v3/users/7be957f2c15c4f6f949b3f284a30624d => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:24:38.381102 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e49d200-5e82-4af9-9e8b-22fda097b420 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:38.382668 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e49d200-5e82-4af9-9e8b-22fda097b420 service glance] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:38.394452 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1346/2681] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:24:38 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:24:38.634018 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f994e1f-4c71-4693-b764-7bd6c61a94d4 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:38.637788 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-2f994e1f-4c71-4693-b764-7bd6c61a94d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:38.781278 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f994e1f-4c71-4693-b764-7bd6c61a94d4 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:38.792985 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dfd58b-b6c1-4781-9d0c-dfe369a04600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:38.793840 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dfd58b-b6c1-4781-9d0c-dfe369a04600 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:38.794116 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dfd58b-b6c1-4781-9d0c-dfe369a04600 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:38.794390 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dfd58b-b6c1-4781-9d0c-dfe369a04600 admin admin] PATH_INFO: `/v3/projects/d0bdb5e8899b4766b797688ad00c7ce0` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:38.795933 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dfd58b-b6c1-4781-9d0c-dfe369a04600 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0bdb5e8899b4766b797688ad00c7ce0)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:38.798987 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dfd58b-b6c1-4781-9d0c-dfe369a04600 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:38.876992 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-44dfd58b-b6c1-4781-9d0c-dfe369a04600 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0bdb5e8899b4766b797688ad00c7ce0', 'request_id': 'req-44dfd58b-b6c1-4781-9d0c-dfe369a04600'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:38.888863 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-44dfd58b-b6c1-4781-9d0c-dfe369a04600 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0bdb5e8899b4766b797688ad00c7ce0', 'request_id': 'req-44dfd58b-b6c1-4781-9d0c-dfe369a04600'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:38.900984 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-44dfd58b-b6c1-4781-9d0c-dfe369a04600 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0bdb5e8899b4766b797688ad00c7ce0', 'request_id': 'req-44dfd58b-b6c1-4781-9d0c-dfe369a04600'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:38.924267 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-44dfd58b-b6c1-4781-9d0c-dfe369a04600 admin admin] The token cache is being invalidate because project d0bdb5e8899b4766b797688ad00c7ce0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81771) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:24:38.925837 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-44dfd58b-b6c1-4781-9d0c-dfe369a04600 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:38.927181 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-44dfd58b-b6c1-4781-9d0c-dfe369a04600 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:38.929396 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1336/2682] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:24:38 2022] DELETE /identity/v3/projects/d0bdb5e8899b4766b797688ad00c7ce0 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:24:38.937096 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c664b9-1c1a-4984-9e42-0378f66d3bc2 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:38.938020 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c664b9-1c1a-4984-9e42-0378f66d3bc2 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:38.938020 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c664b9-1c1a-4984-9e42-0378f66d3bc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:38.987005 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-96c664b9-1c1a-4984-9e42-0378f66d3bc2 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:24:38.990173 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-96c664b9-1c1a-4984-9e42-0378f66d3bc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:39.164521 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dfd58b-b6c1-4781-9d0c-dfe369a04600 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:39.166419 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-44dfd58b-b6c1-4781-9d0c-dfe369a04600 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:39.191524 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1347/2683] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:24:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:24:39.201633 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604afea6-2a62-44d3-b625-cc0233fe7d81 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:39.202047 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604afea6-2a62-44d3-b625-cc0233fe7d81 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:39.203082 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604afea6-2a62-44d3-b625-cc0233fe7d81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:39.233468 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-604afea6-2a62-44d3-b625-cc0233fe7d81 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:24:39.233816 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-604afea6-2a62-44d3-b625-cc0233fe7d81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:39.246569 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1348/2684] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:24:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:24:39.254430 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604afea6-2a62-44d3-b625-cc0233fe7d81 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:39.274508 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604afea6-2a62-44d3-b625-cc0233fe7d81 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:39.282038 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a475d8d3-7325-4d98-ba4d-b5ec00cd0747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:39.282732 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a475d8d3-7325-4d98-ba4d-b5ec00cd0747 admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:39.283015 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a475d8d3-7325-4d98-ba4d-b5ec00cd0747 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:39.283303 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a475d8d3-7325-4d98-ba4d-b5ec00cd0747 admin admin] PATH_INFO: `/v3/domains` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:39.284762 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a475d8d3-7325-4d98-ba4d-b5ec00cd0747 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:24:39.285199 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a475d8d3-7325-4d98-ba4d-b5ec00cd0747 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:39.287165 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a475d8d3-7325-4d98-ba4d-b5ec00cd0747 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:39.287876 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dfd58b-b6c1-4781-9d0c-dfe369a04600 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:39.300164 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb0b152-d5b9-4755-9e84-55467c2ffe21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:39.301588 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb0b152-d5b9-4755-9e84-55467c2ffe21 service neutron] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:39.302094 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb0b152-d5b9-4755-9e84-55467c2ffe21 service neutron] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:39.302547 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb0b152-d5b9-4755-9e84-55467c2ffe21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:39.305536 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-dcb0b152-d5b9-4755-9e84-55467c2ffe21 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:39.309095 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1349/2685] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 6 20:24:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:24:39.314522 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a475d8d3-7325-4d98-ba4d-b5ec00cd0747 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:39.327354 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb0b152-d5b9-4755-9e84-55467c2ffe21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:39.329992 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb0b152-d5b9-4755-9e84-55467c2ffe21 service neutron] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:39.333906 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a475d8d3-7325-4d98-ba4d-b5ec00cd0747 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:39.341485 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fc83b4-599b-45fa-9b4a-80632db57ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:39.342655 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fc83b4-599b-45fa-9b4a-80632db57ac2 admin admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:39.343178 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fc83b4-599b-45fa-9b4a-80632db57ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:39.343706 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fc83b4-599b-45fa-9b4a-80632db57ac2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:39.344904 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fc83b4-599b-45fa-9b4a-80632db57ac2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:39.348984 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1337/2686] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:24:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 20:24:39.349391 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fc83b4-599b-45fa-9b4a-80632db57ac2 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:39.385819 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1350/2687] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:24:39 2022] POST /identity/v3/projects => generated 375 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 20:24:39.390765 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb0b152-d5b9-4755-9e84-55467c2ffe21 service neutron] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:39.403510 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb0b152-d5b9-4755-9e84-55467c2ffe21 service neutron] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:39.429194 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:24:39.493110 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1053e8f-0a1c-4154-b9d5-a932c1612c9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:39.493791 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1053e8f-0a1c-4154-b9d5-a932c1612c9f admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:39.494086 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1053e8f-0a1c-4154-b9d5-a932c1612c9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:39.494355 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1053e8f-0a1c-4154-b9d5-a932c1612c9f admin admin] PATH_INFO: `/v3/users` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:39.495021 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1053e8f-0a1c-4154-b9d5-a932c1612c9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:39.497000 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1053e8f-0a1c-4154-b9d5-a932c1612c9f admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:39.522580 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1338/2688] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:24:39 2022] POST /identity/v3/users => generated 288 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:24:39.528566 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fc83b4-599b-45fa-9b4a-80632db57ac2 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:39.541396 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fc83b4-599b-45fa-9b4a-80632db57ac2 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:39.550114 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27197355-2906-400e-923c-896bf4907ae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:39.550757 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27197355-2906-400e-923c-896bf4907ae9 admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:39.551137 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27197355-2906-400e-923c-896bf4907ae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:39.551449 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27197355-2906-400e-923c-896bf4907ae9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:39.552307 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27197355-2906-400e-923c-896bf4907ae9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:39.554546 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27197355-2906-400e-923c-896bf4907ae9 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:39.563101 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:24:39.570673 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1351/2689] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:24:39 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:24:39.576488 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1053e8f-0a1c-4154-b9d5-a932c1612c9f admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:39.595590 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1053e8f-0a1c-4154-b9d5-a932c1612c9f admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:39.605409 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd4fc60-3ab8-4272-9863-9d7d69a18d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:39.606695 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd4fc60-3ab8-4272-9863-9d7d69a18d6a admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:39.607126 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd4fc60-3ab8-4272-9863-9d7d69a18d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:39.607947 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd4fc60-3ab8-4272-9863-9d7d69a18d6a admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:39.609216 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd4fc60-3ab8-4272-9863-9d7d69a18d6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:39.612957 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd4fc60-3ab8-4272-9863-9d7d69a18d6a admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:39.635969 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1339/2690] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:24:39 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:24:39.646440 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27197355-2906-400e-923c-896bf4907ae9 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:39.663999 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27197355-2906-400e-923c-896bf4907ae9 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:39.676265 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892ace90-1694-4f53-ab80-65f77989f3b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:39.677407 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892ace90-1694-4f53-ab80-65f77989f3b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:39.677931 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892ace90-1694-4f53-ab80-65f77989f3b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:39.678439 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892ace90-1694-4f53-ab80-65f77989f3b8 admin admin] PATH_INFO: `/v3/projects/0b2546fb97eb4db083c5e90af38aa355/users/676f6109261c4b02901003c9399e2c88/roles/4eea39e24b06431aa357053fe7907a3d` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:39.705685 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892ace90-1694-4f53-ab80-65f77989f3b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b2546fb97eb4db083c5e90af38aa355, user_id=676f6109261c4b02901003c9399e2c88, role_id=4eea39e24b06431aa357053fe7907a3d)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:39.709085 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892ace90-1694-4f53-ab80-65f77989f3b8 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:39.728783 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1352/2691] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:24:39 2022] PUT /identity/v3/projects/0b2546fb97eb4db083c5e90af38aa355/users/676f6109261c4b02901003c9399e2c88/roles/4eea39e24b06431aa357053fe7907a3d => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 20:24:39.742836 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ef8168-596c-4a7c-8c59-6ecce05dbf1e None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:39.743234 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ef8168-596c-4a7c-8c59-6ecce05dbf1e None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:39.743758 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ef8168-596c-4a7c-8c59-6ecce05dbf1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:39.802682 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-78ef8168-596c-4a7c-8c59-6ecce05dbf1e None None] MFA Rules not processed for user `676f6109261c4b02901003c9399e2c88`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:24:39.804801 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-78ef8168-596c-4a7c-8c59-6ecce05dbf1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:39.930416 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1340/2692] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:24:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:24:39.944843 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892ace90-1694-4f53-ab80-65f77989f3b8 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:39.949751 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-892ace90-1694-4f53-ab80-65f77989f3b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:40.084713 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892ace90-1694-4f53-ab80-65f77989f3b8 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:40.091445 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb94867a-08bb-43c8-bcef-2106d3ca3025 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5776cbf4f6714b2083c3ebdd6d98cc87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:40.092226 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb94867a-08bb-43c8-bcef-2106d3ca3025 service nova] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:40.092602 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb94867a-08bb-43c8-bcef-2106d3ca3025 service nova] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:40.093172 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb94867a-08bb-43c8-bcef-2106d3ca3025 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:40.103202 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb94867a-08bb-43c8-bcef-2106d3ca3025 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:40.104985 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb94867a-08bb-43c8-bcef-2106d3ca3025 service nova] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:40.118093 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1353/2693] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:24:39 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 20:24:40.444171 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ef8168-596c-4a7c-8c59-6ecce05dbf1e None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:40.557146 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ef8168-596c-4a7c-8c59-6ecce05dbf1e None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:40.577910 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95560d93-c8ef-4709-848f-02066b6c47c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:40.578582 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95560d93-c8ef-4709-848f-02066b6c47c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:40.578865 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95560d93-c8ef-4709-848f-02066b6c47c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:40.579143 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95560d93-c8ef-4709-848f-02066b6c47c6 admin admin] PATH_INFO: `/v3/users/676f6109261c4b02901003c9399e2c88` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:40.582606 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95560d93-c8ef-4709-848f-02066b6c47c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=676f6109261c4b02901003c9399e2c88)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:40.586900 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95560d93-c8ef-4709-848f-02066b6c47c6 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:40.658276 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:24:40.708388 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-95560d93-c8ef-4709-848f-02066b6c47c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '676f6109261c4b02901003c9399e2c88'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:40.723888 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-95560d93-c8ef-4709-848f-02066b6c47c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '676f6109261c4b02901003c9399e2c88'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:40.724959 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-95560d93-c8ef-4709-848f-02066b6c47c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '676f6109261c4b02901003c9399e2c88'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:40.725972 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-95560d93-c8ef-4709-848f-02066b6c47c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '676f6109261c4b02901003c9399e2c88'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:40.737834 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-95560d93-c8ef-4709-848f-02066b6c47c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '676f6109261c4b02901003c9399e2c88'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:40.771888 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1341/2694] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:24:40 2022] DELETE /identity/v3/users/676f6109261c4b02901003c9399e2c88 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 20:24:40.778628 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f797c2-e442-485d-85c0-57db10be47c5 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:40.778924 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f797c2-e442-485d-85c0-57db10be47c5 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:40.779206 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f797c2-e442-485d-85c0-57db10be47c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:40.816915 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-54f797c2-e442-485d-85c0-57db10be47c5 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:24:40.819736 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-54f797c2-e442-485d-85c0-57db10be47c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:40.944977 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1354/2695] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:24:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:24:40.955133 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95560d93-c8ef-4709-848f-02066b6c47c6 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:40.957741 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-95560d93-c8ef-4709-848f-02066b6c47c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:41.025086 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95560d93-c8ef-4709-848f-02066b6c47c6 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:41.029222 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da45991d-08c5-40db-a48c-de131cd52377 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:41.029888 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da45991d-08c5-40db-a48c-de131cd52377 service neutron] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:41.030170 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da45991d-08c5-40db-a48c-de131cd52377 service neutron] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:41.030439 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da45991d-08c5-40db-a48c-de131cd52377 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:41.041721 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da45991d-08c5-40db-a48c-de131cd52377 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:41.043237 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da45991d-08c5-40db-a48c-de131cd52377 service neutron] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:41.057150 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1342/2696] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:24:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:24:41.525215 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f797c2-e442-485d-85c0-57db10be47c5 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:41.528786 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-54f797c2-e442-485d-85c0-57db10be47c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:41.553309 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f797c2-e442-485d-85c0-57db10be47c5 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:41.568226 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e0f017-6c1b-4a3b-a712-8c312d514bf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:41.570647 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e0f017-6c1b-4a3b-a712-8c312d514bf5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:41.571292 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e0f017-6c1b-4a3b-a712-8c312d514bf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:41.571726 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e0f017-6c1b-4a3b-a712-8c312d514bf5 admin admin] PATH_INFO: `/v3/projects/0b2546fb97eb4db083c5e90af38aa355` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:41.573373 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e0f017-6c1b-4a3b-a712-8c312d514bf5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b2546fb97eb4db083c5e90af38aa355)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:41.575832 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e0f017-6c1b-4a3b-a712-8c312d514bf5 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:41.682653 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-f0e0f017-6c1b-4a3b-a712-8c312d514bf5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b2546fb97eb4db083c5e90af38aa355', 'request_id': 'req-f0e0f017-6c1b-4a3b-a712-8c312d514bf5'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:41.700111 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-f0e0f017-6c1b-4a3b-a712-8c312d514bf5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b2546fb97eb4db083c5e90af38aa355', 'request_id': 'req-f0e0f017-6c1b-4a3b-a712-8c312d514bf5'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:41.701982 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:24:41.711467 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-f0e0f017-6c1b-4a3b-a712-8c312d514bf5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b2546fb97eb4db083c5e90af38aa355', 'request_id': 'req-f0e0f017-6c1b-4a3b-a712-8c312d514bf5'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:41.729167 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-f0e0f017-6c1b-4a3b-a712-8c312d514bf5 admin admin] The token cache is being invalidate because project 0b2546fb97eb4db083c5e90af38aa355 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81772) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:24:41.729578 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-f0e0f017-6c1b-4a3b-a712-8c312d514bf5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:41.730442 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-f0e0f017-6c1b-4a3b-a712-8c312d514bf5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:41.732516 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1355/2697] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:24:41 2022] DELETE /identity/v3/projects/0b2546fb97eb4db083c5e90af38aa355 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:24:41.739637 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc970f8f-f522-4d9e-8c07-5a2142901f80 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:41.740124 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc970f8f-f522-4d9e-8c07-5a2142901f80 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:41.740605 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc970f8f-f522-4d9e-8c07-5a2142901f80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:41.789655 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-cc970f8f-f522-4d9e-8c07-5a2142901f80 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:24:41.790989 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-cc970f8f-f522-4d9e-8c07-5a2142901f80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:41.889269 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1343/2698] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:24:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:24:41.898583 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e01839b-75e1-46c8-8ab5-5aba504a90e2 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:41.898909 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e01839b-75e1-46c8-8ab5-5aba504a90e2 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:41.899120 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e01839b-75e1-46c8-8ab5-5aba504a90e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:41.934887 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-9e01839b-75e1-46c8-8ab5-5aba504a90e2 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:24:41.936627 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-9e01839b-75e1-46c8-8ab5-5aba504a90e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:41.947520 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1356/2699] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:24:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:24:41.953068 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc970f8f-f522-4d9e-8c07-5a2142901f80 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:41.988368 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc970f8f-f522-4d9e-8c07-5a2142901f80 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:41.995364 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a3413c-8d26-4874-98bf-c0482e70fc63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:41.996324 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a3413c-8d26-4874-98bf-c0482e70fc63 admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:41.996791 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a3413c-8d26-4874-98bf-c0482e70fc63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:41.997230 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a3413c-8d26-4874-98bf-c0482e70fc63 admin admin] PATH_INFO: `/v3/domains` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:41.998522 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a3413c-8d26-4874-98bf-c0482e70fc63 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:24:41.999046 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a3413c-8d26-4874-98bf-c0482e70fc63 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:42.005969 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a3413c-8d26-4874-98bf-c0482e70fc63 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:42.037091 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1344/2700] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 6 20:24:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:24:42.043271 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e01839b-75e1-46c8-8ab5-5aba504a90e2 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:42.064590 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e01839b-75e1-46c8-8ab5-5aba504a90e2 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:42.071626 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a842e4-138c-46df-a61e-de8f28383f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:42.073255 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a842e4-138c-46df-a61e-de8f28383f11 admin admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:42.073255 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a842e4-138c-46df-a61e-de8f28383f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:42.073724 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a842e4-138c-46df-a61e-de8f28383f11 admin admin] PATH_INFO: `/v3/projects` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:42.074895 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a842e4-138c-46df-a61e-de8f28383f11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:42.078906 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a842e4-138c-46df-a61e-de8f28383f11 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:42.163358 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1357/2701] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:24:42 2022] POST /identity/v3/projects => generated 365 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:24:42.164726 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a3413c-8d26-4874-98bf-c0482e70fc63 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:42.206080 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a3413c-8d26-4874-98bf-c0482e70fc63 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:42.233211 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:24:42.272850 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afae405-6e2d-42c4-8fc3-28187c2b1ad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:42.273958 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afae405-6e2d-42c4-8fc3-28187c2b1ad4 admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:42.274439 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afae405-6e2d-42c4-8fc3-28187c2b1ad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:42.274834 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afae405-6e2d-42c4-8fc3-28187c2b1ad4 admin admin] PATH_INFO: `/v3/users` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:42.275683 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afae405-6e2d-42c4-8fc3-28187c2b1ad4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:42.278450 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afae405-6e2d-42c4-8fc3-28187c2b1ad4 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:42.333193 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1345/2702] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:24:42 2022] POST /identity/v3/users => generated 283 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:24:42.339468 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a842e4-138c-46df-a61e-de8f28383f11 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:42.360217 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a842e4-138c-46df-a61e-de8f28383f11 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:42.366166 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8313c200-121e-4ddb-b633-c9aef47fb5af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:42.366677 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8313c200-121e-4ddb-b633-c9aef47fb5af admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:42.367048 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8313c200-121e-4ddb-b633-c9aef47fb5af admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:42.367306 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8313c200-121e-4ddb-b633-c9aef47fb5af admin admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:42.368052 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8313c200-121e-4ddb-b633-c9aef47fb5af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:42.370203 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8313c200-121e-4ddb-b633-c9aef47fb5af admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:42.393090 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1358/2703] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:24:42 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:24:42.394715 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afae405-6e2d-42c4-8fc3-28187c2b1ad4 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:42.408413 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afae405-6e2d-42c4-8fc3-28187c2b1ad4 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:42.414405 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d144b877-c42b-40f8-bd74-4b743c498abc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:42.415052 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d144b877-c42b-40f8-bd74-4b743c498abc admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:42.415336 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d144b877-c42b-40f8-bd74-4b743c498abc admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:42.416381 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d144b877-c42b-40f8-bd74-4b743c498abc admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:42.416634 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d144b877-c42b-40f8-bd74-4b743c498abc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:42.418804 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d144b877-c42b-40f8-bd74-4b743c498abc admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:42.437667 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1346/2704] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:24:42 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:24:42.442154 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8313c200-121e-4ddb-b633-c9aef47fb5af admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:42.456304 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8313c200-121e-4ddb-b633-c9aef47fb5af admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:42.462761 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644b9b78-3ca4-42ab-9dd7-2ca06539baec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:42.463641 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644b9b78-3ca4-42ab-9dd7-2ca06539baec admin admin] REQUEST_METHOD: `PUT` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:42.464011 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644b9b78-3ca4-42ab-9dd7-2ca06539baec admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:42.464416 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644b9b78-3ca4-42ab-9dd7-2ca06539baec admin admin] PATH_INFO: `/v3/projects/a5110be54cd24b2490daefdcc9c7fdd9/users/1b38656efafc459e9e510a60da7b831d/roles/4eea39e24b06431aa357053fe7907a3d` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:42.491331 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644b9b78-3ca4-42ab-9dd7-2ca06539baec admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5110be54cd24b2490daefdcc9c7fdd9, user_id=1b38656efafc459e9e510a60da7b831d, role_id=4eea39e24b06431aa357053fe7907a3d)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:42.501909 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644b9b78-3ca4-42ab-9dd7-2ca06539baec admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:42.518632 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1359/2705] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:24:42 2022] PUT /identity/v3/projects/a5110be54cd24b2490daefdcc9c7fdd9/users/1b38656efafc459e9e510a60da7b831d/roles/4eea39e24b06431aa357053fe7907a3d => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:24:42.535657 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f739cf-9d67-41f4-8eb9-410dc52b8f2a None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:42.536033 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f739cf-9d67-41f4-8eb9-410dc52b8f2a None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:42.537825 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f739cf-9d67-41f4-8eb9-410dc52b8f2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:42.579897 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-41f739cf-9d67-41f4-8eb9-410dc52b8f2a None None] MFA Rules not processed for user `1b38656efafc459e9e510a60da7b831d`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:24:42.581744 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-41f739cf-9d67-41f4-8eb9-410dc52b8f2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:42.669771 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1347/2706] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:24:42 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:24:42.673770 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644b9b78-3ca4-42ab-9dd7-2ca06539baec admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:42.810955 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644b9b78-3ca4-42ab-9dd7-2ca06539baec admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:42.826689 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aee9bd-9279-4e91-8280-2c49902222fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:42.827058 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aee9bd-9279-4e91-8280-2c49902222fb admin admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:42.827316 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aee9bd-9279-4e91-8280-2c49902222fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:42.827611 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aee9bd-9279-4e91-8280-2c49902222fb admin admin] PATH_INFO: `/v3/projects` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:42.828240 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aee9bd-9279-4e91-8280-2c49902222fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:42.830424 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aee9bd-9279-4e91-8280-2c49902222fb admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:42.870186 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1360/2707] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:24:42 2022] POST /identity/v3/projects => generated 363 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:24:42.871376 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f739cf-9d67-41f4-8eb9-410dc52b8f2a None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:42.888164 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f739cf-9d67-41f4-8eb9-410dc52b8f2a None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:42.983592 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e269a2-9ca5-435f-b977-b479eed62b0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:42.984368 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e269a2-9ca5-435f-b977-b479eed62b0a admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:42.984700 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e269a2-9ca5-435f-b977-b479eed62b0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:42.985073 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e269a2-9ca5-435f-b977-b479eed62b0a admin admin] PATH_INFO: `/v3/users` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:42.985905 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e269a2-9ca5-435f-b977-b479eed62b0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:42.988848 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e269a2-9ca5-435f-b977-b479eed62b0a admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:43.028201 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aee9bd-9279-4e91-8280-2c49902222fb admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:43.034439 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1348/2708] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:24:42 2022] POST /identity/v3/users => generated 288 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:24:43.044693 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aee9bd-9279-4e91-8280-2c49902222fb admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:43.052690 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a0ca6d-926f-4042-960f-ce6039e7ed6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:43.053918 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a0ca6d-926f-4042-960f-ce6039e7ed6d admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:43.054441 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a0ca6d-926f-4042-960f-ce6039e7ed6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:43.054916 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a0ca6d-926f-4042-960f-ce6039e7ed6d admin admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:43.056166 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a0ca6d-926f-4042-960f-ce6039e7ed6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:43.059185 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a0ca6d-926f-4042-960f-ce6039e7ed6d admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:43.079815 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1361/2709] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:24:43 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:24:43.085495 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e269a2-9ca5-435f-b977-b479eed62b0a admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:43.104867 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e269a2-9ca5-435f-b977-b479eed62b0a admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:43.113980 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcb6846-c848-4555-bd08-215e95a28a4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:43.115303 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcb6846-c848-4555-bd08-215e95a28a4f admin admin] REQUEST_METHOD: `PUT` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:43.115903 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcb6846-c848-4555-bd08-215e95a28a4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:43.116571 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcb6846-c848-4555-bd08-215e95a28a4f admin admin] PATH_INFO: `/v3/projects/eab19ca1278d4de9887492b3a8c417cc/users/cc4b0289cff54b53bc2770ca9764a073/roles/e0ab41d8ae22474688b5361bcc75e0f0` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:43.161261 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcb6846-c848-4555-bd08-215e95a28a4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=eab19ca1278d4de9887492b3a8c417cc, user_id=cc4b0289cff54b53bc2770ca9764a073, role_id=e0ab41d8ae22474688b5361bcc75e0f0)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:43.165972 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcb6846-c848-4555-bd08-215e95a28a4f admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:43.201244 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1349/2710] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:24:43 2022] PUT /identity/v3/projects/eab19ca1278d4de9887492b3a8c417cc/users/cc4b0289cff54b53bc2770ca9764a073/roles/e0ab41d8ae22474688b5361bcc75e0f0 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:24:43.203777 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a0ca6d-926f-4042-960f-ce6039e7ed6d admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:43.294923 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a0ca6d-926f-4042-960f-ce6039e7ed6d admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:43.392539 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:24:43.411765 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cf4447-e2b6-4e19-aad4-0f62354e917f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:43.412979 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cf4447-e2b6-4e19-aad4-0f62354e917f admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:43.413916 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cf4447-e2b6-4e19-aad4-0f62354e917f admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:43.414978 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cf4447-e2b6-4e19-aad4-0f62354e917f admin admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:43.416651 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cf4447-e2b6-4e19-aad4-0f62354e917f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:43.419599 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cf4447-e2b6-4e19-aad4-0f62354e917f admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:43.447559 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1362/2711] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:24:43 2022] GET /identity/v3/roles => generated 1420 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:24:43.452856 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcb6846-c848-4555-bd08-215e95a28a4f admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:43.477829 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcb6846-c848-4555-bd08-215e95a28a4f admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:43.509496 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f92ecc0-4977-46ea-a8d7-e7a47b296fe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:43.510567 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f92ecc0-4977-46ea-a8d7-e7a47b296fe3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:43.511018 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f92ecc0-4977-46ea-a8d7-e7a47b296fe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:43.511459 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f92ecc0-4977-46ea-a8d7-e7a47b296fe3 admin admin] PATH_INFO: `/v3/projects/eab19ca1278d4de9887492b3a8c417cc/users/cc4b0289cff54b53bc2770ca9764a073/roles/e0ab41d8ae22474688b5361bcc75e0f0` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:43.514548 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f92ecc0-4977-46ea-a8d7-e7a47b296fe3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eab19ca1278d4de9887492b3a8c417cc, user_id=cc4b0289cff54b53bc2770ca9764a073, role_id=e0ab41d8ae22474688b5361bcc75e0f0)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:43.517914 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f92ecc0-4977-46ea-a8d7-e7a47b296fe3 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:43.523825 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:24:43.540306 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1350/2712] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:24:43 2022] PUT /identity/v3/projects/eab19ca1278d4de9887492b3a8c417cc/users/cc4b0289cff54b53bc2770ca9764a073/roles/e0ab41d8ae22474688b5361bcc75e0f0 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:24:43.561882 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96539a5c-a099-4d0c-9ee7-901a033298a2 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:43.561882 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96539a5c-a099-4d0c-9ee7-901a033298a2 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:43.561882 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96539a5c-a099-4d0c-9ee7-901a033298a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:43.664490 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-96539a5c-a099-4d0c-9ee7-901a033298a2 None None] MFA Rules not processed for user `cc4b0289cff54b53bc2770ca9764a073`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:24:43.666565 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-96539a5c-a099-4d0c-9ee7-901a033298a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:43.834906 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1363/2713] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:24:43 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:24:43.863437 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f92ecc0-4977-46ea-a8d7-e7a47b296fe3 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:43.866933 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-8f92ecc0-4977-46ea-a8d7-e7a47b296fe3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:43.986442 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f92ecc0-4977-46ea-a8d7-e7a47b296fe3 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:44.000225 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16887e6a-ce7d-47cb-b494-780ef67a2ea9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df1bca993bc142908cc7cf85eabd2b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:44.001166 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16887e6a-ce7d-47cb-b494-780ef67a2ea9 service cinder] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:44.001320 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16887e6a-ce7d-47cb-b494-780ef67a2ea9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:44.001707 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16887e6a-ce7d-47cb-b494-780ef67a2ea9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:44.017407 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16887e6a-ce7d-47cb-b494-780ef67a2ea9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:44.019181 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16887e6a-ce7d-47cb-b494-780ef67a2ea9 service cinder] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:44.032780 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1351/2714] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:24:43 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:24:44.227629 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96539a5c-a099-4d0c-9ee7-901a033298a2 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:44.231183 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-96539a5c-a099-4d0c-9ee7-901a033298a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:44.258406 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96539a5c-a099-4d0c-9ee7-901a033298a2 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:44.263562 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d5feae-95fe-4646-8fde-2bf670fe6f09 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df1bca993bc142908cc7cf85eabd2b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:44.264379 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d5feae-95fe-4646-8fde-2bf670fe6f09 service cinder] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:44.264691 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d5feae-95fe-4646-8fde-2bf670fe6f09 service cinder] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:44.264975 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d5feae-95fe-4646-8fde-2bf670fe6f09 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:44.342838 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d5feae-95fe-4646-8fde-2bf670fe6f09 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:44.344558 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d5feae-95fe-4646-8fde-2bf670fe6f09 service cinder] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:44.357185 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1364/2715] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:24:44 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:24:49.930335 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16887e6a-ce7d-47cb-b494-780ef67a2ea9 service cinder] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:49.931839 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-16887e6a-ce7d-47cb-b494-780ef67a2ea9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:50.036863 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16887e6a-ce7d-47cb-b494-780ef67a2ea9 service cinder] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:50.041949 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5e0ddd-64cd-483d-a2c7-f0aca8e8d767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:50.042626 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5e0ddd-64cd-483d-a2c7-f0aca8e8d767 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:50.042972 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5e0ddd-64cd-483d-a2c7-f0aca8e8d767 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:50.043179 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5e0ddd-64cd-483d-a2c7-f0aca8e8d767 admin admin] PATH_INFO: `/v3/users/aaa686e692444eae835ca343926e0595` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:50.044478 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5e0ddd-64cd-483d-a2c7-f0aca8e8d767 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aaa686e692444eae835ca343926e0595)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:50.046726 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5e0ddd-64cd-483d-a2c7-f0aca8e8d767 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:50.123642 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-2b5e0ddd-64cd-483d-a2c7-f0aca8e8d767 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aaa686e692444eae835ca343926e0595'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:50.138865 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-2b5e0ddd-64cd-483d-a2c7-f0aca8e8d767 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aaa686e692444eae835ca343926e0595'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:50.139693 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-2b5e0ddd-64cd-483d-a2c7-f0aca8e8d767 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aaa686e692444eae835ca343926e0595'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:50.140642 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-2b5e0ddd-64cd-483d-a2c7-f0aca8e8d767 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aaa686e692444eae835ca343926e0595'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:50.152550 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-2b5e0ddd-64cd-483d-a2c7-f0aca8e8d767 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aaa686e692444eae835ca343926e0595'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:50.195553 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1352/2716] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:24:49 2022] DELETE /identity/v3/users/aaa686e692444eae835ca343926e0595 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:24:50.310026 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d5feae-95fe-4646-8fde-2bf670fe6f09 service cinder] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:50.312173 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-c9d5feae-95fe-4646-8fde-2bf670fe6f09 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:50.339677 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d5feae-95fe-4646-8fde-2bf670fe6f09 service cinder] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:50.421855 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddbad94-4216-483e-8f06-8a2b39f6a703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:50.422610 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddbad94-4216-483e-8f06-8a2b39f6a703 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:50.422900 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddbad94-4216-483e-8f06-8a2b39f6a703 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:50.423488 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddbad94-4216-483e-8f06-8a2b39f6a703 admin admin] PATH_INFO: `/v3/users/ea970be1bdbb43ff9918b5a19924d835` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:50.426194 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddbad94-4216-483e-8f06-8a2b39f6a703 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea970be1bdbb43ff9918b5a19924d835)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:50.428556 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddbad94-4216-483e-8f06-8a2b39f6a703 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:50.448922 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:24:50.481653 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5e0ddd-64cd-483d-a2c7-f0aca8e8d767 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:50.483960 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-2b5e0ddd-64cd-483d-a2c7-f0aca8e8d767 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:50.498019 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-0ddbad94-4216-483e-8f06-8a2b39f6a703 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea970be1bdbb43ff9918b5a19924d835'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:50.498885 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-0ddbad94-4216-483e-8f06-8a2b39f6a703 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea970be1bdbb43ff9918b5a19924d835'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:50.509545 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5e0ddd-64cd-483d-a2c7-f0aca8e8d767 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:50.516503 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0459661a-d3ac-429d-aa42-16795502e5a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:50.518321 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-0ddbad94-4216-483e-8f06-8a2b39f6a703 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea970be1bdbb43ff9918b5a19924d835'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:50.519374 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-0ddbad94-4216-483e-8f06-8a2b39f6a703 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea970be1bdbb43ff9918b5a19924d835'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:50.526432 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0459661a-d3ac-429d-aa42-16795502e5a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:50.527110 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0459661a-d3ac-429d-aa42-16795502e5a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:50.527504 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0459661a-d3ac-429d-aa42-16795502e5a3 admin admin] PATH_INFO: `/v3/projects/b030ea0d617b4583a2397921c884aaa5` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:50.529755 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0459661a-d3ac-429d-aa42-16795502e5a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b030ea0d617b4583a2397921c884aaa5)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:50.531677 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0459661a-d3ac-429d-aa42-16795502e5a3 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:50.534159 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-0ddbad94-4216-483e-8f06-8a2b39f6a703 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea970be1bdbb43ff9918b5a19924d835'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:50.546961 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:24:50.552485 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1365/2717] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:24:50 2022] DELETE /identity/v3/users/ea970be1bdbb43ff9918b5a19924d835 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 20:24:50.609304 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-0459661a-d3ac-429d-aa42-16795502e5a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b030ea0d617b4583a2397921c884aaa5', 'request_id': 'req-0459661a-d3ac-429d-aa42-16795502e5a3'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:50.617396 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-0459661a-d3ac-429d-aa42-16795502e5a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b030ea0d617b4583a2397921c884aaa5', 'request_id': 'req-0459661a-d3ac-429d-aa42-16795502e5a3'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:50.627131 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-0459661a-d3ac-429d-aa42-16795502e5a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b030ea0d617b4583a2397921c884aaa5', 'request_id': 'req-0459661a-d3ac-429d-aa42-16795502e5a3'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:50.643951 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-0459661a-d3ac-429d-aa42-16795502e5a3 admin admin] The token cache is being invalidate because project b030ea0d617b4583a2397921c884aaa5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81771) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:24:50.644355 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-0459661a-d3ac-429d-aa42-16795502e5a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:50.645139 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-0459661a-d3ac-429d-aa42-16795502e5a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:50.647025 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1353/2718] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:24:50 2022] DELETE /identity/v3/projects/b030ea0d617b4583a2397921c884aaa5 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:24:50.651478 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddbad94-4216-483e-8f06-8a2b39f6a703 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:50.653860 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-0ddbad94-4216-483e-8f06-8a2b39f6a703 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:50.747239 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddbad94-4216-483e-8f06-8a2b39f6a703 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:50.839012 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b98906c-e86d-42b2-a27d-33a32b07cc5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:50.839983 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b98906c-e86d-42b2-a27d-33a32b07cc5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:50.840375 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b98906c-e86d-42b2-a27d-33a32b07cc5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:50.840677 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b98906c-e86d-42b2-a27d-33a32b07cc5b admin admin] PATH_INFO: `/v3/users/3c8660aeec144ec3af7a272dca472f2f` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:50.842485 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b98906c-e86d-42b2-a27d-33a32b07cc5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c8660aeec144ec3af7a272dca472f2f)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:50.845477 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b98906c-e86d-42b2-a27d-33a32b07cc5b admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:50.853880 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0459661a-d3ac-429d-aa42-16795502e5a3 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:50.856234 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-0459661a-d3ac-429d-aa42-16795502e5a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:50.879054 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0459661a-d3ac-429d-aa42-16795502e5a3 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:50.887535 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a85791-de66-4a13-bc8a-219284e459a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:50.888424 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a85791-de66-4a13-bc8a-219284e459a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:50.888815 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a85791-de66-4a13-bc8a-219284e459a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:50.889279 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a85791-de66-4a13-bc8a-219284e459a3 admin admin] PATH_INFO: `/v3/projects/c05cd2d805834013a01d51e4f0528fea` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:50.891035 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a85791-de66-4a13-bc8a-219284e459a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c05cd2d805834013a01d51e4f0528fea)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:50.895749 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a85791-de66-4a13-bc8a-219284e459a3 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:50.939965 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:24:50.947107 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-0b98906c-e86d-42b2-a27d-33a32b07cc5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c8660aeec144ec3af7a272dca472f2f'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:50.948547 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-0b98906c-e86d-42b2-a27d-33a32b07cc5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c8660aeec144ec3af7a272dca472f2f'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:50.974380 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-0b98906c-e86d-42b2-a27d-33a32b07cc5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c8660aeec144ec3af7a272dca472f2f'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:50.975618 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-0b98906c-e86d-42b2-a27d-33a32b07cc5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c8660aeec144ec3af7a272dca472f2f'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:50.984037 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-39a85791-de66-4a13-bc8a-219284e459a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c05cd2d805834013a01d51e4f0528fea', 'request_id': 'req-39a85791-de66-4a13-bc8a-219284e459a3'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:50.990848 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-0b98906c-e86d-42b2-a27d-33a32b07cc5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c8660aeec144ec3af7a272dca472f2f'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:50.996639 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-39a85791-de66-4a13-bc8a-219284e459a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c05cd2d805834013a01d51e4f0528fea', 'request_id': 'req-39a85791-de66-4a13-bc8a-219284e459a3'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:51.010498 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-39a85791-de66-4a13-bc8a-219284e459a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c05cd2d805834013a01d51e4f0528fea', 'request_id': 'req-39a85791-de66-4a13-bc8a-219284e459a3'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:51.016626 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1366/2719] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:24:50 2022] DELETE /identity/v3/users/3c8660aeec144ec3af7a272dca472f2f => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 20:24:51.039951 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-39a85791-de66-4a13-bc8a-219284e459a3 admin admin] The token cache is being invalidate because project c05cd2d805834013a01d51e4f0528fea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81771) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:24:51.040792 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-39a85791-de66-4a13-bc8a-219284e459a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:51.042050 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-39a85791-de66-4a13-bc8a-219284e459a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:51.044576 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1354/2720] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:24:50 2022] DELETE /identity/v3/projects/c05cd2d805834013a01d51e4f0528fea => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:24:51.051375 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b98906c-e86d-42b2-a27d-33a32b07cc5b admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:51.053482 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-0b98906c-e86d-42b2-a27d-33a32b07cc5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:51.153649 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b98906c-e86d-42b2-a27d-33a32b07cc5b admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:51.160951 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a8b063-7317-40bf-8310-728ce9a37717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:51.161650 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a8b063-7317-40bf-8310-728ce9a37717 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:51.161945 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a8b063-7317-40bf-8310-728ce9a37717 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:51.162220 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a8b063-7317-40bf-8310-728ce9a37717 admin admin] PATH_INFO: `/v3/users/b3f39e4c4a2944aa9957bdee39d2d26e` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:51.163754 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a8b063-7317-40bf-8310-728ce9a37717 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3f39e4c4a2944aa9957bdee39d2d26e)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:51.166192 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a8b063-7317-40bf-8310-728ce9a37717 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:51.230571 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-07a8b063-7317-40bf-8310-728ce9a37717 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3f39e4c4a2944aa9957bdee39d2d26e'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:51.231892 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-07a8b063-7317-40bf-8310-728ce9a37717 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3f39e4c4a2944aa9957bdee39d2d26e'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:51.251260 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-07a8b063-7317-40bf-8310-728ce9a37717 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3f39e4c4a2944aa9957bdee39d2d26e'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:51.252211 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-07a8b063-7317-40bf-8310-728ce9a37717 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3f39e4c4a2944aa9957bdee39d2d26e'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:51.263898 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-07a8b063-7317-40bf-8310-728ce9a37717 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3f39e4c4a2944aa9957bdee39d2d26e'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:51.281504 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1367/2721] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:24:51 2022] DELETE /identity/v3/users/b3f39e4c4a2944aa9957bdee39d2d26e => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:24:51.529666 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a85791-de66-4a13-bc8a-219284e459a3 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:51.532103 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-39a85791-de66-4a13-bc8a-219284e459a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:51.649851 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a85791-de66-4a13-bc8a-219284e459a3 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:51.703331 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:24:51.726366 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a8b063-7317-40bf-8310-728ce9a37717 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:51.728674 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-07a8b063-7317-40bf-8310-728ce9a37717 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:51.756615 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d13ff5d-b9f7-4702-88c4-f578b3227ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:51.757971 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d13ff5d-b9f7-4702-88c4-f578b3227ee3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:51.758668 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d13ff5d-b9f7-4702-88c4-f578b3227ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:51.759310 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d13ff5d-b9f7-4702-88c4-f578b3227ee3 admin admin] PATH_INFO: `/v3/projects/99a4bd4b2653468698cd20c18b864920` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:51.761813 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d13ff5d-b9f7-4702-88c4-f578b3227ee3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99a4bd4b2653468698cd20c18b864920)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:51.765947 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d13ff5d-b9f7-4702-88c4-f578b3227ee3 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:51.820985 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:24:51.847732 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-1d13ff5d-b9f7-4702-88c4-f578b3227ee3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99a4bd4b2653468698cd20c18b864920', 'request_id': 'req-1d13ff5d-b9f7-4702-88c4-f578b3227ee3'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:51.856763 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-1d13ff5d-b9f7-4702-88c4-f578b3227ee3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99a4bd4b2653468698cd20c18b864920', 'request_id': 'req-1d13ff5d-b9f7-4702-88c4-f578b3227ee3'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:51.860346 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a8b063-7317-40bf-8310-728ce9a37717 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:51.867845 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-1d13ff5d-b9f7-4702-88c4-f578b3227ee3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99a4bd4b2653468698cd20c18b864920', 'request_id': 'req-1d13ff5d-b9f7-4702-88c4-f578b3227ee3'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:51.883247 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-1d13ff5d-b9f7-4702-88c4-f578b3227ee3 admin admin] The token cache is being invalidate because project 99a4bd4b2653468698cd20c18b864920 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81771) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:24:51.883747 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-1d13ff5d-b9f7-4702-88c4-f578b3227ee3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:51.884660 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-1d13ff5d-b9f7-4702-88c4-f578b3227ee3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:51.886464 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1355/2722] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:24:51 2022] DELETE /identity/v3/projects/99a4bd4b2653468698cd20c18b864920 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:24:51.892449 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a18f0d-072d-4eb8-a3a5-05d58b7d74b6 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:51.893981 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a18f0d-072d-4eb8-a3a5-05d58b7d74b6 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:51.894149 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a18f0d-072d-4eb8-a3a5-05d58b7d74b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:51.921654 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-05a18f0d-072d-4eb8-a3a5-05d58b7d74b6 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:24:51.922944 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-05a18f0d-072d-4eb8-a3a5-05d58b7d74b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:51.992797 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90238bcb-f28d-44f3-a5c3-3191da40ac87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:51.993955 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90238bcb-f28d-44f3-a5c3-3191da40ac87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:51.994463 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90238bcb-f28d-44f3-a5c3-3191da40ac87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:51.994972 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90238bcb-f28d-44f3-a5c3-3191da40ac87 admin admin] PATH_INFO: `/v3/projects/d954cb34eed447b793bb6a493dae972a` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:51.997156 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90238bcb-f28d-44f3-a5c3-3191da40ac87 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d954cb34eed447b793bb6a493dae972a)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:52.001012 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90238bcb-f28d-44f3-a5c3-3191da40ac87 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:52.011236 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1356/2723] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:24:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:24:52.022773 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afa3013-5ba4-49dc-b341-588842a792cf None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:52.023051 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afa3013-5ba4-49dc-b341-588842a792cf None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:52.023318 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afa3013-5ba4-49dc-b341-588842a792cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:52.050401 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-7afa3013-5ba4-49dc-b341-588842a792cf None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:24:52.050930 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-7afa3013-5ba4-49dc-b341-588842a792cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:52.064027 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1357/2724] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:24:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:24:52.071244 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afa3013-5ba4-49dc-b341-588842a792cf None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:52.083248 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afa3013-5ba4-49dc-b341-588842a792cf None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:52.093734 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-90238bcb-f28d-44f3-a5c3-3191da40ac87 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd954cb34eed447b793bb6a493dae972a', 'request_id': 'req-90238bcb-f28d-44f3-a5c3-3191da40ac87'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:52.107778 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-90238bcb-f28d-44f3-a5c3-3191da40ac87 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd954cb34eed447b793bb6a493dae972a', 'request_id': 'req-90238bcb-f28d-44f3-a5c3-3191da40ac87'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:52.120224 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-90238bcb-f28d-44f3-a5c3-3191da40ac87 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd954cb34eed447b793bb6a493dae972a', 'request_id': 'req-90238bcb-f28d-44f3-a5c3-3191da40ac87'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:52.142272 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-90238bcb-f28d-44f3-a5c3-3191da40ac87 admin admin] The token cache is being invalidate because project d954cb34eed447b793bb6a493dae972a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81772) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:24:52.142789 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-90238bcb-f28d-44f3-a5c3-3191da40ac87 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:52.144239 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-90238bcb-f28d-44f3-a5c3-3191da40ac87 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:52.157738 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1368/2725] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:24:51 2022] DELETE /identity/v3/projects/d954cb34eed447b793bb6a493dae972a => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:24:52.207468 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:24:52.225248 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f0f0f3-3b4e-4b58-8cdd-d0f8906f56f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:52.226186 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f0f0f3-3b4e-4b58-8cdd-d0f8906f56f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:52.226505 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f0f0f3-3b4e-4b58-8cdd-d0f8906f56f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:52.226901 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f0f0f3-3b4e-4b58-8cdd-d0f8906f56f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:52.227701 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f0f0f3-3b4e-4b58-8cdd-d0f8906f56f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:24:52.228013 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f0f0f3-3b4e-4b58-8cdd-d0f8906f56f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:52.229870 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f0f0f3-3b4e-4b58-8cdd-d0f8906f56f3 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:52.249669 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1358/2726] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 6 20:24:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:24:52.253598 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90238bcb-f28d-44f3-a5c3-3191da40ac87 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:52.255339 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-90238bcb-f28d-44f3-a5c3-3191da40ac87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:52.271882 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90238bcb-f28d-44f3-a5c3-3191da40ac87 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:52.278847 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bf7976-60ad-46e7-a612-a4d3347f8be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:52.279508 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bf7976-60ad-46e7-a612-a4d3347f8be5 admin admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:52.279822 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bf7976-60ad-46e7-a612-a4d3347f8be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:52.280149 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bf7976-60ad-46e7-a612-a4d3347f8be5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:52.284031 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bf7976-60ad-46e7-a612-a4d3347f8be5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:52.285104 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bf7976-60ad-46e7-a612-a4d3347f8be5 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:52.356616 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1369/2727] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:24:52 2022] POST /identity/v3/projects => generated 373 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:24:52.358442 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f0f0f3-3b4e-4b58-8cdd-d0f8906f56f3 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:52.378419 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f0f0f3-3b4e-4b58-8cdd-d0f8906f56f3 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:52.510856 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4308bc80-8730-475e-8c86-c303c8e13933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:52.514388 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4308bc80-8730-475e-8c86-c303c8e13933 admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:52.514650 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4308bc80-8730-475e-8c86-c303c8e13933 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:52.514929 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4308bc80-8730-475e-8c86-c303c8e13933 admin admin] PATH_INFO: `/v3/users` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:52.515626 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4308bc80-8730-475e-8c86-c303c8e13933 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:52.518744 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4308bc80-8730-475e-8c86-c303c8e13933 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:52.572073 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1359/2728] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:24:52 2022] POST /identity/v3/users => generated 287 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:24:52.578349 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bf7976-60ad-46e7-a612-a4d3347f8be5 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:52.598392 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bf7976-60ad-46e7-a612-a4d3347f8be5 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:52.606545 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035b01c2-6477-4dd3-bcb1-4ccdb2cced6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:52.607542 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035b01c2-6477-4dd3-bcb1-4ccdb2cced6a admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:52.608014 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035b01c2-6477-4dd3-bcb1-4ccdb2cced6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:52.608485 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035b01c2-6477-4dd3-bcb1-4ccdb2cced6a admin admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:52.609689 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035b01c2-6477-4dd3-bcb1-4ccdb2cced6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:52.613226 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035b01c2-6477-4dd3-bcb1-4ccdb2cced6a admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:52.631926 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1370/2729] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:24:52 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 20:24:52.638015 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4308bc80-8730-475e-8c86-c303c8e13933 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:52.651609 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4308bc80-8730-475e-8c86-c303c8e13933 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:52.664471 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0790fcb4-63c0-474d-9df7-696e1fa00c80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:52.664471 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0790fcb4-63c0-474d-9df7-696e1fa00c80 admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:52.664471 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0790fcb4-63c0-474d-9df7-696e1fa00c80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:52.664471 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0790fcb4-63c0-474d-9df7-696e1fa00c80 admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:52.665192 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0790fcb4-63c0-474d-9df7-696e1fa00c80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:52.665327 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0790fcb4-63c0-474d-9df7-696e1fa00c80 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:52.688001 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1360/2730] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:24:52 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 20:24:52.694468 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035b01c2-6477-4dd3-bcb1-4ccdb2cced6a admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:52.713388 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035b01c2-6477-4dd3-bcb1-4ccdb2cced6a admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:52.724935 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da96f938-c3be-4abe-8ed1-6fdbb3ca6937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:52.725970 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da96f938-c3be-4abe-8ed1-6fdbb3ca6937 admin admin] REQUEST_METHOD: `PUT` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:52.726456 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da96f938-c3be-4abe-8ed1-6fdbb3ca6937 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:52.727261 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da96f938-c3be-4abe-8ed1-6fdbb3ca6937 admin admin] PATH_INFO: `/v3/projects/c71a343a0b7b4059bd1b3f54da477eeb/users/8c81235c9ee948929716a1c2ab478904/roles/4eea39e24b06431aa357053fe7907a3d` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:52.750397 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da96f938-c3be-4abe-8ed1-6fdbb3ca6937 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c71a343a0b7b4059bd1b3f54da477eeb, user_id=8c81235c9ee948929716a1c2ab478904, role_id=4eea39e24b06431aa357053fe7907a3d)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:52.755759 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da96f938-c3be-4abe-8ed1-6fdbb3ca6937 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:52.772466 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1371/2731] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:24:52 2022] PUT /identity/v3/projects/c71a343a0b7b4059bd1b3f54da477eeb/users/8c81235c9ee948929716a1c2ab478904/roles/4eea39e24b06431aa357053fe7907a3d => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:24:52.785357 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67357ce-fe02-4f99-9211-7068d4342cb4 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:52.785681 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67357ce-fe02-4f99-9211-7068d4342cb4 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:52.786035 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67357ce-fe02-4f99-9211-7068d4342cb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:52.818889 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-c67357ce-fe02-4f99-9211-7068d4342cb4 None None] MFA Rules not processed for user `8c81235c9ee948929716a1c2ab478904`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:24:52.820250 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-c67357ce-fe02-4f99-9211-7068d4342cb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:52.899871 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1361/2732] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:24:52 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:24:52.919732 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da96f938-c3be-4abe-8ed1-6fdbb3ca6937 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:52.922721 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-da96f938-c3be-4abe-8ed1-6fdbb3ca6937 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:52.961948 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:24:53.019775 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da96f938-c3be-4abe-8ed1-6fdbb3ca6937 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:53.024644 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1424b7c7-09e8-478d-aac8-e9f2e975178e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df1bca993bc142908cc7cf85eabd2b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:53.025751 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1424b7c7-09e8-478d-aac8-e9f2e975178e service cinder] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:53.026302 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1424b7c7-09e8-478d-aac8-e9f2e975178e service cinder] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:53.027241 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1424b7c7-09e8-478d-aac8-e9f2e975178e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:53.042225 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1424b7c7-09e8-478d-aac8-e9f2e975178e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:53.045139 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1424b7c7-09e8-478d-aac8-e9f2e975178e service cinder] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:53.064361 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1372/2733] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:24:52 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:24:53.111210 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67357ce-fe02-4f99-9211-7068d4342cb4 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:53.212280 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:24:53.218486 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67357ce-fe02-4f99-9211-7068d4342cb4 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:53.225553 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabcb202-b355-4ebf-8cec-f069c5220444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:53.226334 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabcb202-b355-4ebf-8cec-f069c5220444 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:53.226923 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabcb202-b355-4ebf-8cec-f069c5220444 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:53.227401 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabcb202-b355-4ebf-8cec-f069c5220444 admin admin] PATH_INFO: `/v3/users/8c81235c9ee948929716a1c2ab478904` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:53.228985 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabcb202-b355-4ebf-8cec-f069c5220444 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c81235c9ee948929716a1c2ab478904)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:53.231353 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabcb202-b355-4ebf-8cec-f069c5220444 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:53.295389 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-eabcb202-b355-4ebf-8cec-f069c5220444 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c81235c9ee948929716a1c2ab478904'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:53.304882 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-eabcb202-b355-4ebf-8cec-f069c5220444 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c81235c9ee948929716a1c2ab478904'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:53.305606 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-eabcb202-b355-4ebf-8cec-f069c5220444 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c81235c9ee948929716a1c2ab478904'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:53.306290 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-eabcb202-b355-4ebf-8cec-f069c5220444 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c81235c9ee948929716a1c2ab478904'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:53.314961 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-eabcb202-b355-4ebf-8cec-f069c5220444 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c81235c9ee948929716a1c2ab478904'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:53.336486 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1362/2734] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:24:53 2022] DELETE /identity/v3/users/8c81235c9ee948929716a1c2ab478904 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:24:53.341710 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65c6f0f-90b4-47b2-8506-bf4e9b1f5e6e None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:53.342053 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65c6f0f-90b4-47b2-8506-bf4e9b1f5e6e None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:53.342338 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65c6f0f-90b4-47b2-8506-bf4e9b1f5e6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:53.371119 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-f65c6f0f-90b4-47b2-8506-bf4e9b1f5e6e None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:24:53.373213 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-f65c6f0f-90b4-47b2-8506-bf4e9b1f5e6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:53.449694 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1373/2735] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:24:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:24:53.463549 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabcb202-b355-4ebf-8cec-f069c5220444 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:53.466149 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-eabcb202-b355-4ebf-8cec-f069c5220444 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:53.538060 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabcb202-b355-4ebf-8cec-f069c5220444 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:53.542138 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b46b56-77e4-4e63-ae74-753bf871fe90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:53.542796 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b46b56-77e4-4e63-ae74-753bf871fe90 service neutron] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:53.543075 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b46b56-77e4-4e63-ae74-753bf871fe90 service neutron] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:53.543349 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b46b56-77e4-4e63-ae74-753bf871fe90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:53.554335 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b46b56-77e4-4e63-ae74-753bf871fe90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:53.555873 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b46b56-77e4-4e63-ae74-753bf871fe90 service neutron] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:53.569150 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1363/2736] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:24:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:24:53.944453 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65c6f0f-90b4-47b2-8506-bf4e9b1f5e6e None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:53.947766 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-f65c6f0f-90b4-47b2-8506-bf4e9b1f5e6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:53.972851 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65c6f0f-90b4-47b2-8506-bf4e9b1f5e6e None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:53.990460 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581b4ffc-a40d-4d9c-9578-e53b5ea2c2ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:53.991960 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581b4ffc-a40d-4d9c-9578-e53b5ea2c2ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:53.992438 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581b4ffc-a40d-4d9c-9578-e53b5ea2c2ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:53.992831 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581b4ffc-a40d-4d9c-9578-e53b5ea2c2ff admin admin] PATH_INFO: `/v3/projects/c71a343a0b7b4059bd1b3f54da477eeb` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:53.995296 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581b4ffc-a40d-4d9c-9578-e53b5ea2c2ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=c71a343a0b7b4059bd1b3f54da477eeb)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:53.998934 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581b4ffc-a40d-4d9c-9578-e53b5ea2c2ff admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:54.065225 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-581b4ffc-a40d-4d9c-9578-e53b5ea2c2ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c71a343a0b7b4059bd1b3f54da477eeb', 'request_id': 'req-581b4ffc-a40d-4d9c-9578-e53b5ea2c2ff'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:54.076278 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-581b4ffc-a40d-4d9c-9578-e53b5ea2c2ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c71a343a0b7b4059bd1b3f54da477eeb', 'request_id': 'req-581b4ffc-a40d-4d9c-9578-e53b5ea2c2ff'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:54.086319 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-581b4ffc-a40d-4d9c-9578-e53b5ea2c2ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c71a343a0b7b4059bd1b3f54da477eeb', 'request_id': 'req-581b4ffc-a40d-4d9c-9578-e53b5ea2c2ff'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:54.100263 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-581b4ffc-a40d-4d9c-9578-e53b5ea2c2ff admin admin] The token cache is being invalidate because project c71a343a0b7b4059bd1b3f54da477eeb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81772) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:24:54.100789 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-581b4ffc-a40d-4d9c-9578-e53b5ea2c2ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:54.101921 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-581b4ffc-a40d-4d9c-9578-e53b5ea2c2ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:54.104188 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1374/2737] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:24:53 2022] DELETE /identity/v3/projects/c71a343a0b7b4059bd1b3f54da477eeb => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 20:24:54.109590 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6261dc2b-e3cf-4b78-857c-401d9d40eff7 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:54.110008 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6261dc2b-e3cf-4b78-857c-401d9d40eff7 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:54.110292 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6261dc2b-e3cf-4b78-857c-401d9d40eff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:54.139724 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-6261dc2b-e3cf-4b78-857c-401d9d40eff7 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:24:54.140990 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-6261dc2b-e3cf-4b78-857c-401d9d40eff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:54.226658 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1364/2738] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:24:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:24:54.239388 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1142b74d-33d1-4cc9-9e63-660ce05d77fe None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:54.239729 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1142b74d-33d1-4cc9-9e63-660ce05d77fe None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:54.240021 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1142b74d-33d1-4cc9-9e63-660ce05d77fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:54.263444 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-1142b74d-33d1-4cc9-9e63-660ce05d77fe None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:24:54.264776 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-1142b74d-33d1-4cc9-9e63-660ce05d77fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:54.273245 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1375/2739] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:24:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:24:54.278476 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6261dc2b-e3cf-4b78-857c-401d9d40eff7 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:54.291468 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6261dc2b-e3cf-4b78-857c-401d9d40eff7 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:54.297122 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01d2fd2-20d1-42a8-bf26-bf7037123452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:54.297803 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01d2fd2-20d1-42a8-bf26-bf7037123452 admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:54.298087 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01d2fd2-20d1-42a8-bf26-bf7037123452 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:54.298364 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01d2fd2-20d1-42a8-bf26-bf7037123452 admin admin] PATH_INFO: `/v3/domains` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:54.299064 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01d2fd2-20d1-42a8-bf26-bf7037123452 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:24:54.299409 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01d2fd2-20d1-42a8-bf26-bf7037123452 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:54.301405 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01d2fd2-20d1-42a8-bf26-bf7037123452 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:54.315642 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1365/2740] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 6 20:24:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:24:54.320439 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1142b74d-33d1-4cc9-9e63-660ce05d77fe None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:54.331697 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1142b74d-33d1-4cc9-9e63-660ce05d77fe None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:54.338677 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0647aa0c-711f-434f-b82b-6d4055477a83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:54.339571 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0647aa0c-711f-434f-b82b-6d4055477a83 admin admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:54.339833 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0647aa0c-711f-434f-b82b-6d4055477a83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:54.340174 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0647aa0c-711f-434f-b82b-6d4055477a83 admin admin] PATH_INFO: `/v3/projects` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:54.341022 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0647aa0c-711f-434f-b82b-6d4055477a83 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:54.343675 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0647aa0c-711f-434f-b82b-6d4055477a83 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:54.379506 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1376/2741] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:24:54 2022] POST /identity/v3/projects => generated 375 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:24:54.384276 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01d2fd2-20d1-42a8-bf26-bf7037123452 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:54.396225 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01d2fd2-20d1-42a8-bf26-bf7037123452 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:54.400535 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:24:54.477840 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738e3320-001b-40e7-8e9c-51350c8dffcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:54.478931 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738e3320-001b-40e7-8e9c-51350c8dffcb admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:54.479358 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738e3320-001b-40e7-8e9c-51350c8dffcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:54.479788 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738e3320-001b-40e7-8e9c-51350c8dffcb admin admin] PATH_INFO: `/v3/users` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:54.480671 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738e3320-001b-40e7-8e9c-51350c8dffcb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:54.483534 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738e3320-001b-40e7-8e9c-51350c8dffcb admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:54.516859 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1366/2742] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:24:54 2022] POST /identity/v3/users => generated 288 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:24:54.523235 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0647aa0c-711f-434f-b82b-6d4055477a83 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:54.535620 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0647aa0c-711f-434f-b82b-6d4055477a83 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:54.541528 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2bd703-d129-4bc5-b6ec-6e305e9a48ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:54.542573 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2bd703-d129-4bc5-b6ec-6e305e9a48ac admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:54.542918 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2bd703-d129-4bc5-b6ec-6e305e9a48ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:54.543247 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2bd703-d129-4bc5-b6ec-6e305e9a48ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:54.543932 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2bd703-d129-4bc5-b6ec-6e305e9a48ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:54.545660 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2bd703-d129-4bc5-b6ec-6e305e9a48ac admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:54.559887 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1377/2743] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:24:54 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:24:54.565390 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738e3320-001b-40e7-8e9c-51350c8dffcb admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:54.579731 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738e3320-001b-40e7-8e9c-51350c8dffcb admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:54.586410 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2962ac97-2e6d-4063-9845-771faf4d3588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:54.587200 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2962ac97-2e6d-4063-9845-771faf4d3588 admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:54.587665 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2962ac97-2e6d-4063-9845-771faf4d3588 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:54.588076 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2962ac97-2e6d-4063-9845-771faf4d3588 admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:54.588973 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2962ac97-2e6d-4063-9845-771faf4d3588 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:54.590919 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2962ac97-2e6d-4063-9845-771faf4d3588 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:54.610321 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1367/2744] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:24:54 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:24:54.615686 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2bd703-d129-4bc5-b6ec-6e305e9a48ac admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:54.632325 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2bd703-d129-4bc5-b6ec-6e305e9a48ac admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:54.640064 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aed879-e4f0-46bd-b6d6-64d6f11372ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:54.640973 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aed879-e4f0-46bd-b6d6-64d6f11372ea admin admin] REQUEST_METHOD: `PUT` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:54.641325 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aed879-e4f0-46bd-b6d6-64d6f11372ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:54.641731 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aed879-e4f0-46bd-b6d6-64d6f11372ea admin admin] PATH_INFO: `/v3/projects/7709cf0cbdb5418d92f698a66780e61f/users/1a7236719669467383c242d0f5a91604/roles/4eea39e24b06431aa357053fe7907a3d` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:54.648085 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:24:54.681986 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7aed879-e4f0-46bd-b6d6-64d6f11372ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=7709cf0cbdb5418d92f698a66780e61f, user_id=1a7236719669467383c242d0f5a91604, role_id=4eea39e24b06431aa357053fe7907a3d)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:54.685398 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7aed879-e4f0-46bd-b6d6-64d6f11372ea admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:54.707777 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1378/2745] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:24:54 2022] PUT /identity/v3/projects/7709cf0cbdb5418d92f698a66780e61f/users/1a7236719669467383c242d0f5a91604/roles/4eea39e24b06431aa357053fe7907a3d => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:24:54.738034 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1159a0ff-e832-44cc-9b4e-409d0bdac657 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:54.738348 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1159a0ff-e832-44cc-9b4e-409d0bdac657 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:54.738544 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1159a0ff-e832-44cc-9b4e-409d0bdac657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:54.775071 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-1159a0ff-e832-44cc-9b4e-409d0bdac657 None None] MFA Rules not processed for user `1a7236719669467383c242d0f5a91604`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:24:54.776340 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-1159a0ff-e832-44cc-9b4e-409d0bdac657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:54.866545 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1368/2746] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:24:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:24:54.884791 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aed879-e4f0-46bd-b6d6-64d6f11372ea admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:54.887240 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-e7aed879-e4f0-46bd-b6d6-64d6f11372ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:54.977271 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aed879-e4f0-46bd-b6d6-64d6f11372ea admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:54.983583 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6258db64-c0ff-4ce1-b3cc-69fa8f267030 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df1bca993bc142908cc7cf85eabd2b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:54.984498 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6258db64-c0ff-4ce1-b3cc-69fa8f267030 service cinder] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:54.985015 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6258db64-c0ff-4ce1-b3cc-69fa8f267030 service cinder] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:54.985455 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6258db64-c0ff-4ce1-b3cc-69fa8f267030 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:55.000150 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6258db64-c0ff-4ce1-b3cc-69fa8f267030 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:55.001713 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6258db64-c0ff-4ce1-b3cc-69fa8f267030 service cinder] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:55.015121 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1379/2747] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:24:54 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:24:55.567665 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1159a0ff-e832-44cc-9b4e-409d0bdac657 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:55.570008 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-1159a0ff-e832-44cc-9b4e-409d0bdac657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:55.674932 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1159a0ff-e832-44cc-9b4e-409d0bdac657 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:55.680610 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1651d48a-a3b4-462e-8008-1152984be9d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:55.681380 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1651d48a-a3b4-462e-8008-1152984be9d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:55.681880 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1651d48a-a3b4-462e-8008-1152984be9d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:55.682311 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1651d48a-a3b4-462e-8008-1152984be9d3 admin admin] PATH_INFO: `/v3/users/1b38656efafc459e9e510a60da7b831d` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:55.683729 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1651d48a-a3b4-462e-8008-1152984be9d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b38656efafc459e9e510a60da7b831d)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:55.686566 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1651d48a-a3b4-462e-8008-1152984be9d3 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:55.706831 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:24:55.756470 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-1651d48a-a3b4-462e-8008-1152984be9d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b38656efafc459e9e510a60da7b831d'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:55.767986 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-1651d48a-a3b4-462e-8008-1152984be9d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b38656efafc459e9e510a60da7b831d'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:55.768656 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-1651d48a-a3b4-462e-8008-1152984be9d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b38656efafc459e9e510a60da7b831d'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:55.769289 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-1651d48a-a3b4-462e-8008-1152984be9d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b38656efafc459e9e510a60da7b831d'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:55.780483 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-1651d48a-a3b4-462e-8008-1152984be9d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b38656efafc459e9e510a60da7b831d'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:55.804730 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1369/2748] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:24:55 2022] DELETE /identity/v3/users/1b38656efafc459e9e510a60da7b831d => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:24:55.810255 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433e6c20-1777-4e33-b48d-1360389ad33a None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:55.810745 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433e6c20-1777-4e33-b48d-1360389ad33a None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:55.811152 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433e6c20-1777-4e33-b48d-1360389ad33a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:55.838385 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-433e6c20-1777-4e33-b48d-1360389ad33a None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:24:55.840378 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-433e6c20-1777-4e33-b48d-1360389ad33a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:55.927591 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1380/2749] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:24:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:24:55.939456 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1651d48a-a3b4-462e-8008-1152984be9d3 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:55.941466 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-1651d48a-a3b4-462e-8008-1152984be9d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:56.013672 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1651d48a-a3b4-462e-8008-1152984be9d3 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:56.020176 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d06ad75-69fd-4c45-b5b0-3cb280841f85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:56.020176 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d06ad75-69fd-4c45-b5b0-3cb280841f85 service neutron] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:56.020176 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d06ad75-69fd-4c45-b5b0-3cb280841f85 service neutron] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:56.020176 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d06ad75-69fd-4c45-b5b0-3cb280841f85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:56.038770 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d06ad75-69fd-4c45-b5b0-3cb280841f85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:56.040835 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d06ad75-69fd-4c45-b5b0-3cb280841f85 service neutron] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:56.055419 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1370/2750] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:24:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 20:24:56.429243 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433e6c20-1777-4e33-b48d-1360389ad33a None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:56.431772 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-433e6c20-1777-4e33-b48d-1360389ad33a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:56.454851 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433e6c20-1777-4e33-b48d-1360389ad33a None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:56.463479 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e06af5-fab1-420a-b74c-712150ddd4ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:56.464581 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e06af5-fab1-420a-b74c-712150ddd4ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:56.465316 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e06af5-fab1-420a-b74c-712150ddd4ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:56.465863 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e06af5-fab1-420a-b74c-712150ddd4ba admin admin] PATH_INFO: `/v3/projects/a5110be54cd24b2490daefdcc9c7fdd9` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:56.467813 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e06af5-fab1-420a-b74c-712150ddd4ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5110be54cd24b2490daefdcc9c7fdd9)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:56.470014 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e06af5-fab1-420a-b74c-712150ddd4ba admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:56.541449 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-f7e06af5-fab1-420a-b74c-712150ddd4ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5110be54cd24b2490daefdcc9c7fdd9', 'request_id': 'req-f7e06af5-fab1-420a-b74c-712150ddd4ba'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:56.559558 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-f7e06af5-fab1-420a-b74c-712150ddd4ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5110be54cd24b2490daefdcc9c7fdd9', 'request_id': 'req-f7e06af5-fab1-420a-b74c-712150ddd4ba'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:56.576346 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-f7e06af5-fab1-420a-b74c-712150ddd4ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5110be54cd24b2490daefdcc9c7fdd9', 'request_id': 'req-f7e06af5-fab1-420a-b74c-712150ddd4ba'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:56.578460 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a33f43-65a1-40c0-82d3-47b9fa106345 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:56.579256 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a33f43-65a1-40c0-82d3-47b9fa106345 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:56.579417 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a33f43-65a1-40c0-82d3-47b9fa106345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:56.593037 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-f7e06af5-fab1-420a-b74c-712150ddd4ba admin admin] The token cache is being invalidate because project a5110be54cd24b2490daefdcc9c7fdd9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81772) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:24:56.593434 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-f7e06af5-fab1-420a-b74c-712150ddd4ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:56.594407 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-f7e06af5-fab1-420a-b74c-712150ddd4ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:56.597222 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1381/2751] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:24:56 2022] DELETE /identity/v3/projects/a5110be54cd24b2490daefdcc9c7fdd9 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 20:24:56.602043 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e06af5-fab1-420a-b74c-712150ddd4ba admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:56.603680 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-f7e06af5-fab1-420a-b74c-712150ddd4ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:56.604554 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-65a33f43-65a1-40c0-82d3-47b9fa106345 None None] MFA Rules not processed for user `1a7236719669467383c242d0f5a91604`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:24:56.605844 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-65a33f43-65a1-40c0-82d3-47b9fa106345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:56.617033 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:24:56.688784 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1371/2752] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:24:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:24:56.716235 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e06af5-fab1-420a-b74c-712150ddd4ba admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:56.722897 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a2b0a3-d550-4a63-9ad4-bee4127a4f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:56.723833 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a2b0a3-d550-4a63-9ad4-bee4127a4f67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:56.724124 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a2b0a3-d550-4a63-9ad4-bee4127a4f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:56.724477 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a2b0a3-d550-4a63-9ad4-bee4127a4f67 admin admin] PATH_INFO: `/v3/users/cc4b0289cff54b53bc2770ca9764a073` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:56.726100 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a2b0a3-d550-4a63-9ad4-bee4127a4f67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc4b0289cff54b53bc2770ca9764a073)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:56.729237 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a2b0a3-d550-4a63-9ad4-bee4127a4f67 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:56.808243 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-c3a2b0a3-d550-4a63-9ad4-bee4127a4f67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc4b0289cff54b53bc2770ca9764a073'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:56.809291 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-c3a2b0a3-d550-4a63-9ad4-bee4127a4f67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc4b0289cff54b53bc2770ca9764a073'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:56.829569 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-c3a2b0a3-d550-4a63-9ad4-bee4127a4f67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc4b0289cff54b53bc2770ca9764a073'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:56.831733 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-c3a2b0a3-d550-4a63-9ad4-bee4127a4f67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc4b0289cff54b53bc2770ca9764a073'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:56.855928 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-c3a2b0a3-d550-4a63-9ad4-bee4127a4f67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc4b0289cff54b53bc2770ca9764a073'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:56.896058 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1382/2753] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:24:56 2022] DELETE /identity/v3/users/cc4b0289cff54b53bc2770ca9764a073 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 20:24:57.405237 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a33f43-65a1-40c0-82d3-47b9fa106345 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:57.436273 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a33f43-65a1-40c0-82d3-47b9fa106345 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:57.533050 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:24:57.579346 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d259d30-1fef-404e-9d31-f4b7c0dcaa77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:57.581267 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d259d30-1fef-404e-9d31-f4b7c0dcaa77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:57.581563 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d259d30-1fef-404e-9d31-f4b7c0dcaa77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:57.582013 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d259d30-1fef-404e-9d31-f4b7c0dcaa77 admin admin] PATH_INFO: `/v3/projects/eab19ca1278d4de9887492b3a8c417cc` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:57.584790 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d259d30-1fef-404e-9d31-f4b7c0dcaa77 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eab19ca1278d4de9887492b3a8c417cc)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:57.588005 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d259d30-1fef-404e-9d31-f4b7c0dcaa77 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:57.751829 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-0d259d30-1fef-404e-9d31-f4b7c0dcaa77 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eab19ca1278d4de9887492b3a8c417cc', 'request_id': 'req-0d259d30-1fef-404e-9d31-f4b7c0dcaa77'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:57.771667 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-0d259d30-1fef-404e-9d31-f4b7c0dcaa77 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eab19ca1278d4de9887492b3a8c417cc', 'request_id': 'req-0d259d30-1fef-404e-9d31-f4b7c0dcaa77'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:57.794716 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-0d259d30-1fef-404e-9d31-f4b7c0dcaa77 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eab19ca1278d4de9887492b3a8c417cc', 'request_id': 'req-0d259d30-1fef-404e-9d31-f4b7c0dcaa77'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:57.823861 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-0d259d30-1fef-404e-9d31-f4b7c0dcaa77 admin admin] The token cache is being invalidate because project eab19ca1278d4de9887492b3a8c417cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81771) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:24:57.824562 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-0d259d30-1fef-404e-9d31-f4b7c0dcaa77 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:57.826344 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-0d259d30-1fef-404e-9d31-f4b7c0dcaa77 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:57.828748 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1372/2754] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:24:57 2022] DELETE /identity/v3/projects/eab19ca1278d4de9887492b3a8c417cc => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:24:57.834945 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b163198-9f4a-4402-b836-a08144c968b2 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:57.836741 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b163198-9f4a-4402-b836-a08144c968b2 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:57.837083 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b163198-9f4a-4402-b836-a08144c968b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:57.862287 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-8b163198-9f4a-4402-b836-a08144c968b2 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:24:57.863597 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-8b163198-9f4a-4402-b836-a08144c968b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:57.988937 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1383/2755] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:24:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:24:57.998681 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa88d2d-f0d7-4e9c-8c83-ad3506d84d0b None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:57.999258 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa88d2d-f0d7-4e9c-8c83-ad3506d84d0b None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:57.999734 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa88d2d-f0d7-4e9c-8c83-ad3506d84d0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:58.032162 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-9aa88d2d-f0d7-4e9c-8c83-ad3506d84d0b None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:24:58.034001 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-9aa88d2d-f0d7-4e9c-8c83-ad3506d84d0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:58.043238 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1373/2756] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:24:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:24:58.048370 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b163198-9f4a-4402-b836-a08144c968b2 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:58.063436 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b163198-9f4a-4402-b836-a08144c968b2 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:58.072073 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277010c7-8279-4f06-8789-808ca463dcf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:58.074073 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277010c7-8279-4f06-8789-808ca463dcf6 admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:58.074509 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277010c7-8279-4f06-8789-808ca463dcf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:58.074901 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277010c7-8279-4f06-8789-808ca463dcf6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:58.075683 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277010c7-8279-4f06-8789-808ca463dcf6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:24:58.076183 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277010c7-8279-4f06-8789-808ca463dcf6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:58.081665 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277010c7-8279-4f06-8789-808ca463dcf6 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:58.106401 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1384/2757] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 6 20:24:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 20:24:58.115110 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa88d2d-f0d7-4e9c-8c83-ad3506d84d0b None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:58.136350 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa88d2d-f0d7-4e9c-8c83-ad3506d84d0b None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:58.146253 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8b2c60-8661-4ec2-bbcd-db1a761c8405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:58.147126 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8b2c60-8661-4ec2-bbcd-db1a761c8405 admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:58.147470 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8b2c60-8661-4ec2-bbcd-db1a761c8405 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:58.147810 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8b2c60-8661-4ec2-bbcd-db1a761c8405 admin admin] PATH_INFO: `/v3/projects` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:58.148758 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8b2c60-8661-4ec2-bbcd-db1a761c8405 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:58.151118 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8b2c60-8661-4ec2-bbcd-db1a761c8405 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:58.200992 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1374/2758] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:24:58 2022] POST /identity/v3/projects => generated 387 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:24:58.206471 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277010c7-8279-4f06-8789-808ca463dcf6 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:58.220264 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277010c7-8279-4f06-8789-808ca463dcf6 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:58.238458 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:24:58.319635 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e8f5be-add8-434f-b986-4acb548aa27a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:58.320706 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e8f5be-add8-434f-b986-4acb548aa27a admin admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:58.321205 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e8f5be-add8-434f-b986-4acb548aa27a admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:58.322287 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e8f5be-add8-434f-b986-4acb548aa27a admin admin] PATH_INFO: `/v3/users` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:58.323808 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e8f5be-add8-434f-b986-4acb548aa27a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:58.327780 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e8f5be-add8-434f-b986-4acb548aa27a admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:58.363509 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1385/2759] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:24:58 2022] POST /identity/v3/users => generated 294 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:24:58.368040 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8b2c60-8661-4ec2-bbcd-db1a761c8405 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:58.383412 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8b2c60-8661-4ec2-bbcd-db1a761c8405 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:58.388800 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7003b86-1244-4644-8822-5fe22aa848b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:58.389558 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7003b86-1244-4644-8822-5fe22aa848b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:58.390023 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7003b86-1244-4644-8822-5fe22aa848b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:58.390393 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7003b86-1244-4644-8822-5fe22aa848b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:58.391221 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7003b86-1244-4644-8822-5fe22aa848b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:58.393718 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7003b86-1244-4644-8822-5fe22aa848b8 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:58.407582 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1375/2760] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:24:58 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:24:58.413169 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e8f5be-add8-434f-b986-4acb548aa27a admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:58.425912 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e8f5be-add8-434f-b986-4acb548aa27a admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:58.433375 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752332a4-be3a-4297-958b-f2a044625f5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:58.434511 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752332a4-be3a-4297-958b-f2a044625f5e admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:58.435006 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752332a4-be3a-4297-958b-f2a044625f5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:58.435523 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752332a4-be3a-4297-958b-f2a044625f5e admin admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:58.436473 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752332a4-be3a-4297-958b-f2a044625f5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:58.438292 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752332a4-be3a-4297-958b-f2a044625f5e admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:58.452308 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1386/2761] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:24:58 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:24:58.457653 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7003b86-1244-4644-8822-5fe22aa848b8 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:58.472222 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7003b86-1244-4644-8822-5fe22aa848b8 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:58.478479 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b505f38-f256-4b5d-b42f-bcd55d698f7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:58.479509 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b505f38-f256-4b5d-b42f-bcd55d698f7a admin admin] REQUEST_METHOD: `PUT` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:58.480068 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b505f38-f256-4b5d-b42f-bcd55d698f7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:58.480679 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b505f38-f256-4b5d-b42f-bcd55d698f7a admin admin] PATH_INFO: `/v3/projects/415b7a1c5da343c3b245632f7f799bab/users/af32266bd719405bbc5d87cd0d7f1e03/roles/4eea39e24b06431aa357053fe7907a3d` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:58.509064 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b505f38-f256-4b5d-b42f-bcd55d698f7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=415b7a1c5da343c3b245632f7f799bab, user_id=af32266bd719405bbc5d87cd0d7f1e03, role_id=4eea39e24b06431aa357053fe7907a3d)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:58.512542 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b505f38-f256-4b5d-b42f-bcd55d698f7a admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:58.537765 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1376/2762] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:24:58 2022] PUT /identity/v3/projects/415b7a1c5da343c3b245632f7f799bab/users/af32266bd719405bbc5d87cd0d7f1e03/roles/4eea39e24b06431aa357053fe7907a3d => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:24:58.545921 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7b5baf-4466-4236-819e-dc4fb300ac22 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:58.546748 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7b5baf-4466-4236-819e-dc4fb300ac22 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:58.548332 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7b5baf-4466-4236-819e-dc4fb300ac22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:58.620332 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-fe7b5baf-4466-4236-819e-dc4fb300ac22 None None] MFA Rules not processed for user `af32266bd719405bbc5d87cd0d7f1e03`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:24:58.622649 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-fe7b5baf-4466-4236-819e-dc4fb300ac22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:58.742772 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1387/2763] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:24:58 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:24:58.747525 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b505f38-f256-4b5d-b42f-bcd55d698f7a admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:58.766577 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7b5baf-4466-4236-819e-dc4fb300ac22 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:58.769840 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-fe7b5baf-4466-4236-819e-dc4fb300ac22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:58.853598 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b505f38-f256-4b5d-b42f-bcd55d698f7a admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:58.859783 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e9f56d-4535-44b6-902c-63615edf6c1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:58.860533 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e9f56d-4535-44b6-902c-63615edf6c1e admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:58.860836 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e9f56d-4535-44b6-902c-63615edf6c1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:58.861565 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e9f56d-4535-44b6-902c-63615edf6c1e admin admin] PATH_INFO: `/v3/projects` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:58.862563 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e9f56d-4535-44b6-902c-63615edf6c1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:58.864746 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e9f56d-4535-44b6-902c-63615edf6c1e admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:58.872049 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7b5baf-4466-4236-819e-dc4fb300ac22 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:58.882348 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f896486-7239-4826-869d-64afcd52276e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:58.883252 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f896486-7239-4826-869d-64afcd52276e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:58.883586 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f896486-7239-4826-869d-64afcd52276e admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:58.883874 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f896486-7239-4826-869d-64afcd52276e admin admin] PATH_INFO: `/v3/users/4dd41c8e6382492d8f77c3799b215ba1` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:58.884996 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:24:58.887955 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f896486-7239-4826-869d-64afcd52276e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4dd41c8e6382492d8f77c3799b215ba1)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:58.890324 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f896486-7239-4826-869d-64afcd52276e admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:58.915263 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1377/2764] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:24:58 2022] POST /identity/v3/projects => generated 387 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:24:58.917214 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e9f56d-4535-44b6-902c-63615edf6c1e admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:58.936574 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:24:58.967362 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-4f896486-7239-4826-869d-64afcd52276e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4dd41c8e6382492d8f77c3799b215ba1'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:58.968109 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-4f896486-7239-4826-869d-64afcd52276e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4dd41c8e6382492d8f77c3799b215ba1'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:58.984998 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-4f896486-7239-4826-869d-64afcd52276e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4dd41c8e6382492d8f77c3799b215ba1'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:58.985856 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-4f896486-7239-4826-869d-64afcd52276e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4dd41c8e6382492d8f77c3799b215ba1'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:58.997043 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-4f896486-7239-4826-869d-64afcd52276e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4dd41c8e6382492d8f77c3799b215ba1'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:24:59.016301 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1388/2765] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:24:58 2022] DELETE /identity/v3/users/4dd41c8e6382492d8f77c3799b215ba1 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:24:59.021680 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5735cc67-c09f-4aa0-996b-07e51fe7e302 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:59.021993 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5735cc67-c09f-4aa0-996b-07e51fe7e302 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:59.022362 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5735cc67-c09f-4aa0-996b-07e51fe7e302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:59.039710 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e9f56d-4535-44b6-902c-63615edf6c1e admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:59.049693 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-5735cc67-c09f-4aa0-996b-07e51fe7e302 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:24:59.051940 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-5735cc67-c09f-4aa0-996b-07e51fe7e302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:59.147752 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5dd18a-ca70-44f1-a3cc-dd4c90269f4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:59.148484 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5dd18a-ca70-44f1-a3cc-dd4c90269f4a admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:59.148932 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5dd18a-ca70-44f1-a3cc-dd4c90269f4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:59.149207 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5dd18a-ca70-44f1-a3cc-dd4c90269f4a admin admin] PATH_INFO: `/v3/users` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:59.149923 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5dd18a-ca70-44f1-a3cc-dd4c90269f4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:59.152451 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5dd18a-ca70-44f1-a3cc-dd4c90269f4a admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:59.177789 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1389/2766] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:24:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:24:59.191241 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5735cc67-c09f-4aa0-996b-07e51fe7e302 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:59.193991 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-5735cc67-c09f-4aa0-996b-07e51fe7e302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:59.195296 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1378/2767] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:24:58 2022] POST /identity/v3/users => generated 294 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:24:59.202071 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5dd18a-ca70-44f1-a3cc-dd4c90269f4a admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:59.206728 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-6a5dd18a-ca70-44f1-a3cc-dd4c90269f4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:59.232258 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5dd18a-ca70-44f1-a3cc-dd4c90269f4a admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:59.242891 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba87cf6-b1cd-4625-b244-19a3fd5d0787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:59.243863 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba87cf6-b1cd-4625-b244-19a3fd5d0787 admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:59.244521 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba87cf6-b1cd-4625-b244-19a3fd5d0787 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:59.245028 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba87cf6-b1cd-4625-b244-19a3fd5d0787 admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:59.246312 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba87cf6-b1cd-4625-b244-19a3fd5d0787 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:59.249515 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba87cf6-b1cd-4625-b244-19a3fd5d0787 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:59.265959 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1379/2768] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:24:59 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:24:59.272649 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba87cf6-b1cd-4625-b244-19a3fd5d0787 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:59.291286 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba87cf6-b1cd-4625-b244-19a3fd5d0787 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:59.298121 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ced8447-2670-4fa2-991a-6576ec019e8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:59.300549 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ced8447-2670-4fa2-991a-6576ec019e8c admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:59.301000 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ced8447-2670-4fa2-991a-6576ec019e8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:59.301458 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ced8447-2670-4fa2-991a-6576ec019e8c admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:59.302752 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ced8447-2670-4fa2-991a-6576ec019e8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:59.305316 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ced8447-2670-4fa2-991a-6576ec019e8c admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:59.306108 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5735cc67-c09f-4aa0-996b-07e51fe7e302 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:59.311330 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621b9c94-6134-4b3b-bca5-a6fdfcaed821 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:59.312112 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621b9c94-6134-4b3b-bca5-a6fdfcaed821 service neutron] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:59.312588 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621b9c94-6134-4b3b-bca5-a6fdfcaed821 service neutron] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:59.312921 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621b9c94-6134-4b3b-bca5-a6fdfcaed821 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:59.320131 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1380/2769] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:24:59 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:24:59.326864 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ced8447-2670-4fa2-991a-6576ec019e8c admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:59.327618 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621b9c94-6134-4b3b-bca5-a6fdfcaed821 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:59.329439 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621b9c94-6134-4b3b-bca5-a6fdfcaed821 service neutron] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:59.345414 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1390/2770] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:24:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:24:59.348713 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ced8447-2670-4fa2-991a-6576ec019e8c admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:59.357590 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab253cd-9cd3-486c-ad20-84c4632cf3be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:59.358521 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab253cd-9cd3-486c-ad20-84c4632cf3be admin admin] REQUEST_METHOD: `PUT` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:59.358933 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab253cd-9cd3-486c-ad20-84c4632cf3be admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:59.359337 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab253cd-9cd3-486c-ad20-84c4632cf3be admin admin] PATH_INFO: `/v3/projects/8c2ad0a5ea8f4d1bac4936a3c1e5ea5f/users/df94707bf7464cd29838e9a4a42387c3/roles/4eea39e24b06431aa357053fe7907a3d` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:59.384597 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab253cd-9cd3-486c-ad20-84c4632cf3be admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c2ad0a5ea8f4d1bac4936a3c1e5ea5f, user_id=df94707bf7464cd29838e9a4a42387c3, role_id=4eea39e24b06431aa357053fe7907a3d)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:59.387843 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab253cd-9cd3-486c-ad20-84c4632cf3be admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:59.406265 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1381/2771] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:24:59 2022] PUT /identity/v3/projects/8c2ad0a5ea8f4d1bac4936a3c1e5ea5f/users/df94707bf7464cd29838e9a4a42387c3/roles/4eea39e24b06431aa357053fe7907a3d => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:24:59.419328 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cc84ef-bb02-4826-a57d-da9a3a12eb54 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:59.419718 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cc84ef-bb02-4826-a57d-da9a3a12eb54 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:59.420044 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cc84ef-bb02-4826-a57d-da9a3a12eb54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:59.468146 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-59cc84ef-bb02-4826-a57d-da9a3a12eb54 None None] MFA Rules not processed for user `df94707bf7464cd29838e9a4a42387c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:24:59.471288 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-59cc84ef-bb02-4826-a57d-da9a3a12eb54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:59.526639 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:24:59.574807 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1391/2772] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:24:59 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:24:59.582250 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab253cd-9cd3-486c-ad20-84c4632cf3be admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:59.675713 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:24:59.704294 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab253cd-9cd3-486c-ad20-84c4632cf3be admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:59.711687 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d008685-5981-470f-94ac-b8cfc622eaac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:59.711687 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d008685-5981-470f-94ac-b8cfc622eaac admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:59.712230 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d008685-5981-470f-94ac-b8cfc622eaac admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:59.712230 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d008685-5981-470f-94ac-b8cfc622eaac admin admin] PATH_INFO: `/v3/projects` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:59.713058 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d008685-5981-470f-94ac-b8cfc622eaac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:59.715736 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d008685-5981-470f-94ac-b8cfc622eaac admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:24:59.759682 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1382/2773] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:24:59 2022] POST /identity/v3/projects => generated 381 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:24:59.766300 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cc84ef-bb02-4826-a57d-da9a3a12eb54 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:59.791926 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cc84ef-bb02-4826-a57d-da9a3a12eb54 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:24:59.887976 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d008685-5981-470f-94ac-b8cfc622eaac admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:24:59.894437 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-6d008685-5981-470f-94ac-b8cfc622eaac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:24:59.931071 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29bb271-4b69-4f61-9bbc-c4b784f7a6aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:24:59.932388 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29bb271-4b69-4f61-9bbc-c4b784f7a6aa admin admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:24:59.932849 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29bb271-4b69-4f61-9bbc-c4b784f7a6aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:24:59.933492 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29bb271-4b69-4f61-9bbc-c4b784f7a6aa admin admin] PATH_INFO: `/v3/users` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:24:59.935013 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29bb271-4b69-4f61-9bbc-c4b784f7a6aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:24:59.939242 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29bb271-4b69-4f61-9bbc-c4b784f7a6aa admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:00.040269 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1392/2774] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:24:59 2022] POST /identity/v3/users => generated 297 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:25:00.042061 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29bb271-4b69-4f61-9bbc-c4b784f7a6aa admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:00.077015 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d008685-5981-470f-94ac-b8cfc622eaac admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:00.083546 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc64b7d7-a0d9-42fa-9f4b-b25d9595aa66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:00.084387 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc64b7d7-a0d9-42fa-9f4b-b25d9595aa66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:00.084745 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc64b7d7-a0d9-42fa-9f4b-b25d9595aa66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:00.085097 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc64b7d7-a0d9-42fa-9f4b-b25d9595aa66 admin admin] PATH_INFO: `/v3/projects/7c54bded329541a8955eeda44c2006b3` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:00.086622 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc64b7d7-a0d9-42fa-9f4b-b25d9595aa66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c54bded329541a8955eeda44c2006b3)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:00.091646 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29bb271-4b69-4f61-9bbc-c4b784f7a6aa admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:00.095522 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc64b7d7-a0d9-42fa-9f4b-b25d9595aa66 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:00.129328 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8d6b8f-193f-4e8a-afb2-2f8d960b7125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:00.130370 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8d6b8f-193f-4e8a-afb2-2f8d960b7125 admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:00.130838 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8d6b8f-193f-4e8a-afb2-2f8d960b7125 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:00.131298 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8d6b8f-193f-4e8a-afb2-2f8d960b7125 admin admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:00.132446 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8d6b8f-193f-4e8a-afb2-2f8d960b7125 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:00.135523 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8d6b8f-193f-4e8a-afb2-2f8d960b7125 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:00.181825 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1393/2775] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:25:00 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 20:25:00.187215 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8d6b8f-193f-4e8a-afb2-2f8d960b7125 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:00.204296 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-cc64b7d7-a0d9-42fa-9f4b-b25d9595aa66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c54bded329541a8955eeda44c2006b3', 'request_id': 'req-cc64b7d7-a0d9-42fa-9f4b-b25d9595aa66'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:00.206916 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8d6b8f-193f-4e8a-afb2-2f8d960b7125 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:00.218940 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-cc64b7d7-a0d9-42fa-9f4b-b25d9595aa66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c54bded329541a8955eeda44c2006b3', 'request_id': 'req-cc64b7d7-a0d9-42fa-9f4b-b25d9595aa66'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:00.233213 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-cc64b7d7-a0d9-42fa-9f4b-b25d9595aa66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c54bded329541a8955eeda44c2006b3', 'request_id': 'req-cc64b7d7-a0d9-42fa-9f4b-b25d9595aa66'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:00.259000 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-cc64b7d7-a0d9-42fa-9f4b-b25d9595aa66 admin admin] The token cache is being invalidate because project 7c54bded329541a8955eeda44c2006b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81771) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:25:00.259314 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-cc64b7d7-a0d9-42fa-9f4b-b25d9595aa66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:00.260172 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-cc64b7d7-a0d9-42fa-9f4b-b25d9595aa66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:00.261881 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1383/2776] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:24:59 2022] DELETE /identity/v3/projects/7c54bded329541a8955eeda44c2006b3 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 20:25:00.311255 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5f8d3f-8f22-479c-a0b8-ec8e3dd8d5ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:00.312013 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5f8d3f-8f22-479c-a0b8-ec8e3dd8d5ae admin admin] REQUEST_METHOD: `PUT` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:00.312333 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5f8d3f-8f22-479c-a0b8-ec8e3dd8d5ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:00.312624 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5f8d3f-8f22-479c-a0b8-ec8e3dd8d5ae admin admin] PATH_INFO: `/v3/projects/062b3043b48240f89c41c439677d9e9c/users/035fddb06f864902b08d08e3602e3d58/roles/e0ab41d8ae22474688b5361bcc75e0f0` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:00.337129 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5f8d3f-8f22-479c-a0b8-ec8e3dd8d5ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=062b3043b48240f89c41c439677d9e9c, user_id=035fddb06f864902b08d08e3602e3d58, role_id=e0ab41d8ae22474688b5361bcc75e0f0)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:00.341162 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5f8d3f-8f22-479c-a0b8-ec8e3dd8d5ae admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:00.359486 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1394/2777] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:25:00 2022] PUT /identity/v3/projects/062b3043b48240f89c41c439677d9e9c/users/035fddb06f864902b08d08e3602e3d58/roles/e0ab41d8ae22474688b5361bcc75e0f0 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:25:00.365352 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc64b7d7-a0d9-42fa-9f4b-b25d9595aa66 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:00.368061 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-cc64b7d7-a0d9-42fa-9f4b-b25d9595aa66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:25:00.388682 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc64b7d7-a0d9-42fa-9f4b-b25d9595aa66 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:00.507057 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea915c8c-a9b8-473f-8d4a-a3d85af7af8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:00.508017 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea915c8c-a9b8-473f-8d4a-a3d85af7af8c admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:00.508440 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea915c8c-a9b8-473f-8d4a-a3d85af7af8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:00.508873 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea915c8c-a9b8-473f-8d4a-a3d85af7af8c admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:00.510219 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea915c8c-a9b8-473f-8d4a-a3d85af7af8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:00.512981 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea915c8c-a9b8-473f-8d4a-a3d85af7af8c admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:00.515867 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:25:00.548051 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1384/2778] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:25:00 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:25:00.555521 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5f8d3f-8f22-479c-a0b8-ec8e3dd8d5ae admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:00.574518 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5f8d3f-8f22-479c-a0b8-ec8e3dd8d5ae admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:00.582348 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3161d3-bf51-468a-b6c5-ec7e4182760e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:00.583153 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3161d3-bf51-468a-b6c5-ec7e4182760e admin admin] REQUEST_METHOD: `PUT` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:00.583463 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3161d3-bf51-468a-b6c5-ec7e4182760e admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:00.583778 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3161d3-bf51-468a-b6c5-ec7e4182760e admin admin] PATH_INFO: `/v3/projects/062b3043b48240f89c41c439677d9e9c/users/035fddb06f864902b08d08e3602e3d58/roles/e0ab41d8ae22474688b5361bcc75e0f0` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:00.586338 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3161d3-bf51-468a-b6c5-ec7e4182760e admin admin] RBAC: Authorizing `identity:create_grant(project_id=062b3043b48240f89c41c439677d9e9c, user_id=035fddb06f864902b08d08e3602e3d58, role_id=e0ab41d8ae22474688b5361bcc75e0f0)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:00.589295 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3161d3-bf51-468a-b6c5-ec7e4182760e admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:00.610280 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1395/2779] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:25:00 2022] PUT /identity/v3/projects/062b3043b48240f89c41c439677d9e9c/users/035fddb06f864902b08d08e3602e3d58/roles/e0ab41d8ae22474688b5361bcc75e0f0 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:25:00.623783 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb93b5f-9aa7-42fb-bac8-d9ba9e274e78 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:00.624267 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb93b5f-9aa7-42fb-bac8-d9ba9e274e78 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:00.624762 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb93b5f-9aa7-42fb-bac8-d9ba9e274e78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:00.680990 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-7eb93b5f-9aa7-42fb-bac8-d9ba9e274e78 None None] MFA Rules not processed for user `035fddb06f864902b08d08e3602e3d58`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:25:00.683442 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-7eb93b5f-9aa7-42fb-bac8-d9ba9e274e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:25:00.821088 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1385/2780] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:25:00 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:25:00.840423 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3161d3-bf51-468a-b6c5-ec7e4182760e admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:00.843736 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-ac3161d3-bf51-468a-b6c5-ec7e4182760e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:25:00.890120 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:25:00.936768 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3161d3-bf51-468a-b6c5-ec7e4182760e admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:00.942321 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139df5d7-8317-4bfc-86f9-f3749660da1f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df1bca993bc142908cc7cf85eabd2b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:00.943085 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139df5d7-8317-4bfc-86f9-f3749660da1f service cinder] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:00.943603 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139df5d7-8317-4bfc-86f9-f3749660da1f service cinder] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:00.944114 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139df5d7-8317-4bfc-86f9-f3749660da1f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:00.955404 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139df5d7-8317-4bfc-86f9-f3749660da1f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:00.957049 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139df5d7-8317-4bfc-86f9-f3749660da1f service cinder] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:00.968524 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1396/2781] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:25:00 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:25:01.884715 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb93b5f-9aa7-42fb-bac8-d9ba9e274e78 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:01.886519 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-7eb93b5f-9aa7-42fb-bac8-d9ba9e274e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:25:01.913069 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb93b5f-9aa7-42fb-bac8-d9ba9e274e78 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:01.919249 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d0781e-a514-4278-a3fc-11e0faeef268 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df1bca993bc142908cc7cf85eabd2b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:01.920023 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d0781e-a514-4278-a3fc-11e0faeef268 service cinder] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:01.920353 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d0781e-a514-4278-a3fc-11e0faeef268 service cinder] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:01.920810 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d0781e-a514-4278-a3fc-11e0faeef268 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:01.922992 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-b0d0781e-a514-4278-a3fc-11e0faeef268 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:25:02.028774 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d0781e-a514-4278-a3fc-11e0faeef268 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:02.031861 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d0781e-a514-4278-a3fc-11e0faeef268 service cinder] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:02.050810 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1386/2782] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:25:01 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 20:25:06.468974 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139df5d7-8317-4bfc-86f9-f3749660da1f service cinder] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:06.481006 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139df5d7-8317-4bfc-86f9-f3749660da1f service cinder] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:06.485494 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e31963-4630-4018-a031-453f9124acb3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df1bca993bc142908cc7cf85eabd2b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:06.486160 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e31963-4630-4018-a031-453f9124acb3 service cinder] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:06.486420 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e31963-4630-4018-a031-453f9124acb3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:06.486787 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e31963-4630-4018-a031-453f9124acb3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:06.488896 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-c0e31963-4630-4018-a031-453f9124acb3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:25:06.574097 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e31963-4630-4018-a031-453f9124acb3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:06.576049 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e31963-4630-4018-a031-453f9124acb3 service cinder] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:06.591431 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1397/2783] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:25:06 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:25:09.279019 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d0781e-a514-4278-a3fc-11e0faeef268 service cinder] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:09.384274 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d0781e-a514-4278-a3fc-11e0faeef268 service cinder] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:09.389545 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10734e07-7ecf-4992-ba48-cceb9e8871e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:09.390470 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10734e07-7ecf-4992-ba48-cceb9e8871e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:09.390799 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10734e07-7ecf-4992-ba48-cceb9e8871e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:09.391079 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10734e07-7ecf-4992-ba48-cceb9e8871e3 admin admin] PATH_INFO: `/v3/users/af32266bd719405bbc5d87cd0d7f1e03` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:09.392635 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10734e07-7ecf-4992-ba48-cceb9e8871e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af32266bd719405bbc5d87cd0d7f1e03)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:09.395320 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10734e07-7ecf-4992-ba48-cceb9e8871e3 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:09.425436 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:25:09.459252 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-10734e07-7ecf-4992-ba48-cceb9e8871e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af32266bd719405bbc5d87cd0d7f1e03'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:09.469505 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-10734e07-7ecf-4992-ba48-cceb9e8871e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af32266bd719405bbc5d87cd0d7f1e03'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:09.470260 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-10734e07-7ecf-4992-ba48-cceb9e8871e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af32266bd719405bbc5d87cd0d7f1e03'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:09.471809 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-10734e07-7ecf-4992-ba48-cceb9e8871e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af32266bd719405bbc5d87cd0d7f1e03'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:09.486107 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-10734e07-7ecf-4992-ba48-cceb9e8871e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af32266bd719405bbc5d87cd0d7f1e03'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:09.521461 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1387/2784] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:25:09 2022] DELETE /identity/v3/users/af32266bd719405bbc5d87cd0d7f1e03 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:25:09.528072 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d0d6a4-db11-4aba-a67c-7c2a017f340b None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:09.528941 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d0d6a4-db11-4aba-a67c-7c2a017f340b None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:09.528941 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d0d6a4-db11-4aba-a67c-7c2a017f340b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:09.555187 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-81d0d6a4-db11-4aba-a67c-7c2a017f340b None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:25:09.563604 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-81d0d6a4-db11-4aba-a67c-7c2a017f340b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:25:09.657018 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1398/2785] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:25:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:25:09.667772 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10734e07-7ecf-4992-ba48-cceb9e8871e3 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:09.669467 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-10734e07-7ecf-4992-ba48-cceb9e8871e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:25:09.749342 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10734e07-7ecf-4992-ba48-cceb9e8871e3 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:09.753596 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89274572-1606-4a43-ab4c-af9554d9823e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:09.754261 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89274572-1606-4a43-ab4c-af9554d9823e service neutron] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:09.754674 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89274572-1606-4a43-ab4c-af9554d9823e service neutron] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:09.754984 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89274572-1606-4a43-ab4c-af9554d9823e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:09.766764 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89274572-1606-4a43-ab4c-af9554d9823e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:09.769003 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89274572-1606-4a43-ab4c-af9554d9823e service neutron] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:09.781721 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1388/2786] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:25:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:25:10.160975 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d0d6a4-db11-4aba-a67c-7c2a017f340b None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:10.163358 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-81d0d6a4-db11-4aba-a67c-7c2a017f340b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:25:10.187060 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d0d6a4-db11-4aba-a67c-7c2a017f340b None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:10.195801 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e081e26-95a7-4a64-9011-cdf91a076f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:10.196896 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e081e26-95a7-4a64-9011-cdf91a076f0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:10.197501 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e081e26-95a7-4a64-9011-cdf91a076f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:10.197906 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e081e26-95a7-4a64-9011-cdf91a076f0e admin admin] PATH_INFO: `/v3/projects/415b7a1c5da343c3b245632f7f799bab` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:10.200118 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e081e26-95a7-4a64-9011-cdf91a076f0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=415b7a1c5da343c3b245632f7f799bab)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:10.203913 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e081e26-95a7-4a64-9011-cdf91a076f0e admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:10.289075 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-2e081e26-95a7-4a64-9011-cdf91a076f0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '415b7a1c5da343c3b245632f7f799bab', 'request_id': 'req-2e081e26-95a7-4a64-9011-cdf91a076f0e'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:10.299900 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-2e081e26-95a7-4a64-9011-cdf91a076f0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '415b7a1c5da343c3b245632f7f799bab', 'request_id': 'req-2e081e26-95a7-4a64-9011-cdf91a076f0e'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:10.308975 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-2e081e26-95a7-4a64-9011-cdf91a076f0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '415b7a1c5da343c3b245632f7f799bab', 'request_id': 'req-2e081e26-95a7-4a64-9011-cdf91a076f0e'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:10.310922 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:25:10.331864 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-2e081e26-95a7-4a64-9011-cdf91a076f0e admin admin] The token cache is being invalidate because project 415b7a1c5da343c3b245632f7f799bab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81772) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:25:10.331864 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-2e081e26-95a7-4a64-9011-cdf91a076f0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:10.331864 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-2e081e26-95a7-4a64-9011-cdf91a076f0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:10.334084 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1399/2787] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:25:10 2022] DELETE /identity/v3/projects/415b7a1c5da343c3b245632f7f799bab => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:25:10.339178 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89274572-1606-4a43-ab4c-af9554d9823e service neutron] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:10.352377 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89274572-1606-4a43-ab4c-af9554d9823e service neutron] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:10.379397 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e081e26-95a7-4a64-9011-cdf91a076f0e admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:10.381566 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-2e081e26-95a7-4a64-9011-cdf91a076f0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:25:10.436748 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eb4cba-038d-4fa4-846f-4d9765ecb14b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:10.438102 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eb4cba-038d-4fa4-846f-4d9765ecb14b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:10.438551 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eb4cba-038d-4fa4-846f-4d9765ecb14b admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:10.439030 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eb4cba-038d-4fa4-846f-4d9765ecb14b admin admin] PATH_INFO: `/v3/users/df94707bf7464cd29838e9a4a42387c3` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:10.441474 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76eb4cba-038d-4fa4-846f-4d9765ecb14b admin admin] RBAC: Authorizing `identity:delete_user(user_id=df94707bf7464cd29838e9a4a42387c3)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:10.445478 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76eb4cba-038d-4fa4-846f-4d9765ecb14b admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:10.469304 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e081e26-95a7-4a64-9011-cdf91a076f0e admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:10.475296 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbe7652-1b05-49f8-8dd2-1294877e7486 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:10.476391 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbe7652-1b05-49f8-8dd2-1294877e7486 service neutron] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:10.476938 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbe7652-1b05-49f8-8dd2-1294877e7486 service neutron] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:10.477423 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbe7652-1b05-49f8-8dd2-1294877e7486 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:10.480721 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-2bbe7652-1b05-49f8-8dd2-1294877e7486 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:25:10.523726 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:25:10.529119 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-76eb4cba-038d-4fa4-846f-4d9765ecb14b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df94707bf7464cd29838e9a4a42387c3'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:10.542738 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-76eb4cba-038d-4fa4-846f-4d9765ecb14b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df94707bf7464cd29838e9a4a42387c3'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:10.544560 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-76eb4cba-038d-4fa4-846f-4d9765ecb14b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df94707bf7464cd29838e9a4a42387c3'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:10.545498 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-76eb4cba-038d-4fa4-846f-4d9765ecb14b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df94707bf7464cd29838e9a4a42387c3'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:10.559027 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-76eb4cba-038d-4fa4-846f-4d9765ecb14b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df94707bf7464cd29838e9a4a42387c3'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:10.583816 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1389/2788] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:25:10 2022] DELETE /identity/v3/users/df94707bf7464cd29838e9a4a42387c3 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:25:10.594212 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbe7652-1b05-49f8-8dd2-1294877e7486 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:10.596663 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbe7652-1b05-49f8-8dd2-1294877e7486 service neutron] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:10.618159 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1400/2789] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:25:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 20:25:11.148272 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eb4cba-038d-4fa4-846f-4d9765ecb14b admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:11.151439 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-76eb4cba-038d-4fa4-846f-4d9765ecb14b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:25:11.276486 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eb4cba-038d-4fa4-846f-4d9765ecb14b admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:11.286033 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e32375-d258-438b-a9c3-344fad46ffd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:11.286780 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e32375-d258-438b-a9c3-344fad46ffd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:11.287145 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e32375-d258-438b-a9c3-344fad46ffd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:11.287501 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e32375-d258-438b-a9c3-344fad46ffd9 admin admin] PATH_INFO: `/v3/projects/8c2ad0a5ea8f4d1bac4936a3c1e5ea5f` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:11.289089 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e32375-d258-438b-a9c3-344fad46ffd9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c2ad0a5ea8f4d1bac4936a3c1e5ea5f)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:11.291558 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e32375-d258-438b-a9c3-344fad46ffd9 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:11.354954 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-64e32375-d258-438b-a9c3-344fad46ffd9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c2ad0a5ea8f4d1bac4936a3c1e5ea5f', 'request_id': 'req-64e32375-d258-438b-a9c3-344fad46ffd9'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:11.363919 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-64e32375-d258-438b-a9c3-344fad46ffd9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c2ad0a5ea8f4d1bac4936a3c1e5ea5f', 'request_id': 'req-64e32375-d258-438b-a9c3-344fad46ffd9'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:11.374293 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-64e32375-d258-438b-a9c3-344fad46ffd9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c2ad0a5ea8f4d1bac4936a3c1e5ea5f', 'request_id': 'req-64e32375-d258-438b-a9c3-344fad46ffd9'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:11.392923 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-64e32375-d258-438b-a9c3-344fad46ffd9 admin admin] The token cache is being invalidate because project 8c2ad0a5ea8f4d1bac4936a3c1e5ea5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81771) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:25:11.393337 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-64e32375-d258-438b-a9c3-344fad46ffd9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:11.394482 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-64e32375-d258-438b-a9c3-344fad46ffd9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:11.397051 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1390/2790] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:25:11 2022] DELETE /identity/v3/projects/8c2ad0a5ea8f4d1bac4936a3c1e5ea5f => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:25:11.401710 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbe7652-1b05-49f8-8dd2-1294877e7486 service neutron] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:11.403664 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-2bbe7652-1b05-49f8-8dd2-1294877e7486 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:25:11.419718 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbe7652-1b05-49f8-8dd2-1294877e7486 service neutron] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:11.492957 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0eb2320-0141-4fbd-901b-199eae087cf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:11.493649 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eb2320-0141-4fbd-901b-199eae087cf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:11.494039 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eb2320-0141-4fbd-901b-199eae087cf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:11.494324 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eb2320-0141-4fbd-901b-199eae087cf1 admin admin] PATH_INFO: `/v3/users/035fddb06f864902b08d08e3602e3d58` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:11.495815 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0eb2320-0141-4fbd-901b-199eae087cf1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=035fddb06f864902b08d08e3602e3d58)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:11.498297 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0eb2320-0141-4fbd-901b-199eae087cf1 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:11.546340 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:DELETE=7,keystone:SELECT=93 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:25:11.563066 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-e0eb2320-0141-4fbd-901b-199eae087cf1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '035fddb06f864902b08d08e3602e3d58'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:11.564068 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-e0eb2320-0141-4fbd-901b-199eae087cf1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '035fddb06f864902b08d08e3602e3d58'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:11.580546 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-e0eb2320-0141-4fbd-901b-199eae087cf1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '035fddb06f864902b08d08e3602e3d58'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:11.581332 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-e0eb2320-0141-4fbd-901b-199eae087cf1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '035fddb06f864902b08d08e3602e3d58'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:11.592098 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-e0eb2320-0141-4fbd-901b-199eae087cf1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '035fddb06f864902b08d08e3602e3d58'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:11.609118 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1401/2791] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:25:11 2022] DELETE /identity/v3/users/035fddb06f864902b08d08e3602e3d58 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:25:11.991105 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e32375-d258-438b-a9c3-344fad46ffd9 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:11.992923 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-64e32375-d258-438b-a9c3-344fad46ffd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:25:12.014920 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e32375-d258-438b-a9c3-344fad46ffd9 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:12.086430 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:25:12.105676 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4593212a-45d3-4aee-88f8-52774982a31d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:12.106405 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4593212a-45d3-4aee-88f8-52774982a31d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:12.106737 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4593212a-45d3-4aee-88f8-52774982a31d admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:12.107065 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4593212a-45d3-4aee-88f8-52774982a31d admin admin] PATH_INFO: `/v3/projects/062b3043b48240f89c41c439677d9e9c` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:12.108726 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4593212a-45d3-4aee-88f8-52774982a31d admin admin] RBAC: Authorizing `identity:delete_project(project_id=062b3043b48240f89c41c439677d9e9c)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:12.111437 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4593212a-45d3-4aee-88f8-52774982a31d admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:12.177822 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-4593212a-45d3-4aee-88f8-52774982a31d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '062b3043b48240f89c41c439677d9e9c', 'request_id': 'req-4593212a-45d3-4aee-88f8-52774982a31d'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:12.191696 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-4593212a-45d3-4aee-88f8-52774982a31d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '062b3043b48240f89c41c439677d9e9c', 'request_id': 'req-4593212a-45d3-4aee-88f8-52774982a31d'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:12.208347 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-4593212a-45d3-4aee-88f8-52774982a31d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '062b3043b48240f89c41c439677d9e9c', 'request_id': 'req-4593212a-45d3-4aee-88f8-52774982a31d'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:12.224624 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-4593212a-45d3-4aee-88f8-52774982a31d admin admin] The token cache is being invalidate because project 062b3043b48240f89c41c439677d9e9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81771) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:25:12.224962 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-4593212a-45d3-4aee-88f8-52774982a31d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:12.225776 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-4593212a-45d3-4aee-88f8-52774982a31d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:12.227297 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1391/2792] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:25:11 2022] DELETE /identity/v3/projects/062b3043b48240f89c41c439677d9e9c => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:25:12.234233 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15179547-3e45-4776-914d-746d7c204fc7 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:12.234508 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15179547-3e45-4776-914d-746d7c204fc7 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:12.234791 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15179547-3e45-4776-914d-746d7c204fc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:12.264011 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-15179547-3e45-4776-914d-746d7c204fc7 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:25:12.266379 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-15179547-3e45-4776-914d-746d7c204fc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:25:12.350531 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1402/2793] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:25:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:25:12.360862 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bac7175-ca3d-4c88-b9f8-1147fbb20f98 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:12.361717 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bac7175-ca3d-4c88-b9f8-1147fbb20f98 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:12.362101 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bac7175-ca3d-4c88-b9f8-1147fbb20f98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:12.396536 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-9bac7175-ca3d-4c88-b9f8-1147fbb20f98 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:25:12.398729 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-9bac7175-ca3d-4c88-b9f8-1147fbb20f98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:25:12.413747 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1392/2794] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:25:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:25:12.419534 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15179547-3e45-4776-914d-746d7c204fc7 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:12.432045 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15179547-3e45-4776-914d-746d7c204fc7 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:12.438418 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1aa4216-bdda-4a71-b442-8daa5b93ca9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:12.439732 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aa4216-bdda-4a71-b442-8daa5b93ca9e admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:12.440087 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aa4216-bdda-4a71-b442-8daa5b93ca9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:12.440548 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aa4216-bdda-4a71-b442-8daa5b93ca9e admin admin] PATH_INFO: `/v3/domains` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:12.441362 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1aa4216-bdda-4a71-b442-8daa5b93ca9e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:25:12.441826 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1aa4216-bdda-4a71-b442-8daa5b93ca9e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:12.443976 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1aa4216-bdda-4a71-b442-8daa5b93ca9e admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:12.460458 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1403/2795] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 6 20:25:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:25:12.465533 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bac7175-ca3d-4c88-b9f8-1147fbb20f98 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:12.478293 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bac7175-ca3d-4c88-b9f8-1147fbb20f98 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:12.483676 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121eb4ea-1527-421d-b2d0-ceb206c8a522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:12.484328 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121eb4ea-1527-421d-b2d0-ceb206c8a522 admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:12.484612 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121eb4ea-1527-421d-b2d0-ceb206c8a522 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:12.484890 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121eb4ea-1527-421d-b2d0-ceb206c8a522 admin admin] PATH_INFO: `/v3/projects` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:12.485575 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121eb4ea-1527-421d-b2d0-ceb206c8a522 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:12.487798 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121eb4ea-1527-421d-b2d0-ceb206c8a522 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:12.525409 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1393/2796] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:25:12 2022] POST /identity/v3/projects => generated 379 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 20:25:12.530171 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1aa4216-bdda-4a71-b442-8daa5b93ca9e admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:12.542766 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1aa4216-bdda-4a71-b442-8daa5b93ca9e admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:12.667303 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793fc559-eb45-4b09-97d6-d191824495b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:12.668686 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793fc559-eb45-4b09-97d6-d191824495b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:12.669210 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793fc559-eb45-4b09-97d6-d191824495b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:12.669740 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793fc559-eb45-4b09-97d6-d191824495b2 admin admin] PATH_INFO: `/v3/users` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:12.671004 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793fc559-eb45-4b09-97d6-d191824495b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:12.675545 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793fc559-eb45-4b09-97d6-d191824495b2 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:12.714807 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1404/2797] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:25:12 2022] POST /identity/v3/users => generated 290 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:25:12.719957 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121eb4ea-1527-421d-b2d0-ceb206c8a522 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:12.739926 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121eb4ea-1527-421d-b2d0-ceb206c8a522 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:12.748914 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acd8a86-60fc-47fc-93d3-ca6ea4030bf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:12.750248 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acd8a86-60fc-47fc-93d3-ca6ea4030bf2 admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:12.750722 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acd8a86-60fc-47fc-93d3-ca6ea4030bf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:12.751486 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acd8a86-60fc-47fc-93d3-ca6ea4030bf2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:12.753059 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acd8a86-60fc-47fc-93d3-ca6ea4030bf2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:12.756614 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acd8a86-60fc-47fc-93d3-ca6ea4030bf2 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:12.792816 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1394/2798] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:25:12 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:25:12.797057 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793fc559-eb45-4b09-97d6-d191824495b2 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:12.826730 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793fc559-eb45-4b09-97d6-d191824495b2 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:12.838124 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf35339-0b47-4d10-b0a9-89cc0679638f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:12.839305 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf35339-0b47-4d10-b0a9-89cc0679638f admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:12.839859 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf35339-0b47-4d10-b0a9-89cc0679638f admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:12.840349 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf35339-0b47-4d10-b0a9-89cc0679638f admin admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:12.844095 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf35339-0b47-4d10-b0a9-89cc0679638f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:12.846183 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf35339-0b47-4d10-b0a9-89cc0679638f admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:12.869754 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1405/2799] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:25:12 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:25:12.876570 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acd8a86-60fc-47fc-93d3-ca6ea4030bf2 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:12.898093 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acd8a86-60fc-47fc-93d3-ca6ea4030bf2 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:12.908600 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191cc0fd-0fc1-49f5-9264-f3eb9c420c34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:12.909816 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191cc0fd-0fc1-49f5-9264-f3eb9c420c34 admin admin] REQUEST_METHOD: `PUT` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:12.910310 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191cc0fd-0fc1-49f5-9264-f3eb9c420c34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:12.911602 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191cc0fd-0fc1-49f5-9264-f3eb9c420c34 admin admin] PATH_INFO: `/v3/projects/eff5a2e6f99d4210b1580e5ad20868e1/users/c6e731ecdfc244408867e35ef2310fb4/roles/4eea39e24b06431aa357053fe7907a3d` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:12.947842 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191cc0fd-0fc1-49f5-9264-f3eb9c420c34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eff5a2e6f99d4210b1580e5ad20868e1, user_id=c6e731ecdfc244408867e35ef2310fb4, role_id=4eea39e24b06431aa357053fe7907a3d)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:12.951479 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191cc0fd-0fc1-49f5-9264-f3eb9c420c34 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:12.973651 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1395/2800] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:25:12 2022] PUT /identity/v3/projects/eff5a2e6f99d4210b1580e5ad20868e1/users/c6e731ecdfc244408867e35ef2310fb4/roles/4eea39e24b06431aa357053fe7907a3d => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:25:12.995091 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc27934-2679-4070-9637-0a4f76aafbac None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:12.995423 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc27934-2679-4070-9637-0a4f76aafbac None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:12.996414 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc27934-2679-4070-9637-0a4f76aafbac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:13.039794 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-0bc27934-2679-4070-9637-0a4f76aafbac None None] MFA Rules not processed for user `c6e731ecdfc244408867e35ef2310fb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:25:13.041322 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-0bc27934-2679-4070-9637-0a4f76aafbac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:25:13.048458 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:25:13.184934 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1406/2801] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:25:12 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:25:13.190760 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191cc0fd-0fc1-49f5-9264-f3eb9c420c34 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:13.308258 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191cc0fd-0fc1-49f5-9264-f3eb9c420c34 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:13.320175 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4912c45d-d9be-43ee-a6c4-75ef63200415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:13.321038 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4912c45d-d9be-43ee-a6c4-75ef63200415 admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:13.321325 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4912c45d-d9be-43ee-a6c4-75ef63200415 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:13.321854 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4912c45d-d9be-43ee-a6c4-75ef63200415 admin admin] PATH_INFO: `/v3/projects` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:13.322384 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4912c45d-d9be-43ee-a6c4-75ef63200415 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:13.324628 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4912c45d-d9be-43ee-a6c4-75ef63200415 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:13.381443 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1396/2802] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:25:13 2022] POST /identity/v3/projects => generated 379 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:25:13.385857 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc27934-2679-4070-9637-0a4f76aafbac None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:13.414691 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc27934-2679-4070-9637-0a4f76aafbac None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:13.569921 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c45ef1f-f37c-44e9-a4b3-7b550acda0a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:13.570644 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c45ef1f-f37c-44e9-a4b3-7b550acda0a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:13.570967 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c45ef1f-f37c-44e9-a4b3-7b550acda0a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:13.571277 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c45ef1f-f37c-44e9-a4b3-7b550acda0a5 admin admin] PATH_INFO: `/v3/users` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:13.572017 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c45ef1f-f37c-44e9-a4b3-7b550acda0a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:13.574196 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c45ef1f-f37c-44e9-a4b3-7b550acda0a5 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:13.611664 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1407/2803] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:25:13 2022] POST /identity/v3/users => generated 290 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:25:13.614606 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4912c45d-d9be-43ee-a6c4-75ef63200415 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:13.629839 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:25:13.638258 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4912c45d-d9be-43ee-a6c4-75ef63200415 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:13.657045 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948b3720-172b-4cd0-923a-8e8be2e8ace9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:13.658251 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948b3720-172b-4cd0-923a-8e8be2e8ace9 admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:13.658735 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948b3720-172b-4cd0-923a-8e8be2e8ace9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:13.659229 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948b3720-172b-4cd0-923a-8e8be2e8ace9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:13.660398 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948b3720-172b-4cd0-923a-8e8be2e8ace9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:13.663602 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948b3720-172b-4cd0-923a-8e8be2e8ace9 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:13.694128 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1397/2804] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:25:13 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:25:13.695575 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c45ef1f-f37c-44e9-a4b3-7b550acda0a5 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:13.709238 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c45ef1f-f37c-44e9-a4b3-7b550acda0a5 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:13.714966 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bf6a87-4a91-41b9-872c-6e7c07bc0a87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:13.715738 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bf6a87-4a91-41b9-872c-6e7c07bc0a87 admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:13.716088 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bf6a87-4a91-41b9-872c-6e7c07bc0a87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:13.716565 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bf6a87-4a91-41b9-872c-6e7c07bc0a87 admin admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:13.717354 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bf6a87-4a91-41b9-872c-6e7c07bc0a87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:13.719462 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bf6a87-4a91-41b9-872c-6e7c07bc0a87 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:13.742456 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1408/2805] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:25:13 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:25:13.749146 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948b3720-172b-4cd0-923a-8e8be2e8ace9 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:13.768412 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948b3720-172b-4cd0-923a-8e8be2e8ace9 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:13.779498 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5a7bd1-d118-4a9c-bbdc-22272c550f57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:13.780664 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5a7bd1-d118-4a9c-bbdc-22272c550f57 admin admin] REQUEST_METHOD: `PUT` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:13.781135 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5a7bd1-d118-4a9c-bbdc-22272c550f57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:13.781661 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5a7bd1-d118-4a9c-bbdc-22272c550f57 admin admin] PATH_INFO: `/v3/projects/924a36b66a3f4082b35cc4770e2e9cac/users/43ebba6fee7d4113b805455b1ba9d43d/roles/4eea39e24b06431aa357053fe7907a3d` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:13.810596 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5a7bd1-d118-4a9c-bbdc-22272c550f57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=924a36b66a3f4082b35cc4770e2e9cac, user_id=43ebba6fee7d4113b805455b1ba9d43d, role_id=4eea39e24b06431aa357053fe7907a3d)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:13.816726 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5a7bd1-d118-4a9c-bbdc-22272c550f57 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:13.836953 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1398/2806] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:25:13 2022] PUT /identity/v3/projects/924a36b66a3f4082b35cc4770e2e9cac/users/43ebba6fee7d4113b805455b1ba9d43d/roles/4eea39e24b06431aa357053fe7907a3d => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:25:13.851357 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36032013-535d-4f4b-a1d2-a61c076713a7 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:13.851654 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36032013-535d-4f4b-a1d2-a61c076713a7 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:13.851952 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36032013-535d-4f4b-a1d2-a61c076713a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:13.886334 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-36032013-535d-4f4b-a1d2-a61c076713a7 None None] MFA Rules not processed for user `43ebba6fee7d4113b805455b1ba9d43d`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:25:13.887626 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-36032013-535d-4f4b-a1d2-a61c076713a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:25:13.982885 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1409/2807] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:25:13 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:25:13.989094 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5a7bd1-d118-4a9c-bbdc-22272c550f57 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:14.092169 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5a7bd1-d118-4a9c-bbdc-22272c550f57 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:14.101580 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f09fec-4ada-4cd4-bd95-c0083639574a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:14.103194 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f09fec-4ada-4cd4-bd95-c0083639574a admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:14.103194 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f09fec-4ada-4cd4-bd95-c0083639574a admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:14.103587 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f09fec-4ada-4cd4-bd95-c0083639574a admin admin] PATH_INFO: `/v3/projects` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:14.104251 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f09fec-4ada-4cd4-bd95-c0083639574a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:14.106620 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f09fec-4ada-4cd4-bd95-c0083639574a admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:14.149342 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1399/2808] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:25:13 2022] POST /identity/v3/projects => generated 379 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:25:14.155030 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36032013-535d-4f4b-a1d2-a61c076713a7 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:14.172569 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36032013-535d-4f4b-a1d2-a61c076713a7 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:14.190779 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:25:14.281257 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235c29e7-1805-457d-a09e-6f39e29fe59a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:14.282334 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235c29e7-1805-457d-a09e-6f39e29fe59a admin admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:14.282897 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235c29e7-1805-457d-a09e-6f39e29fe59a admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:14.283386 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235c29e7-1805-457d-a09e-6f39e29fe59a admin admin] PATH_INFO: `/v3/users` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:14.284606 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235c29e7-1805-457d-a09e-6f39e29fe59a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:14.288464 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235c29e7-1805-457d-a09e-6f39e29fe59a admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:14.322272 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1410/2809] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:25:14 2022] POST /identity/v3/users => generated 296 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:25:14.327733 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f09fec-4ada-4cd4-bd95-c0083639574a admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:14.347131 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f09fec-4ada-4cd4-bd95-c0083639574a admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:14.354395 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34b1e53-1a0e-4e42-bd92-9f67b081a29c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:14.355468 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34b1e53-1a0e-4e42-bd92-9f67b081a29c admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:14.355938 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34b1e53-1a0e-4e42-bd92-9f67b081a29c admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:14.356404 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34b1e53-1a0e-4e42-bd92-9f67b081a29c admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:14.357541 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34b1e53-1a0e-4e42-bd92-9f67b081a29c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:14.360607 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34b1e53-1a0e-4e42-bd92-9f67b081a29c admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:14.381834 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1400/2810] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:25:14 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:25:14.386636 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235c29e7-1805-457d-a09e-6f39e29fe59a admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:14.399576 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235c29e7-1805-457d-a09e-6f39e29fe59a admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:14.406706 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44a5947-212d-4c70-bb7d-1de6ad7f9ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:14.407978 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44a5947-212d-4c70-bb7d-1de6ad7f9ea7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:14.408519 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44a5947-212d-4c70-bb7d-1de6ad7f9ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:14.409031 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44a5947-212d-4c70-bb7d-1de6ad7f9ea7 admin admin] PATH_INFO: `/v3/projects/b5f1432bd4d44aba8f8a279da573b2b3/users/34d42062701a4e10a8685652ae431458/roles/e0ab41d8ae22474688b5361bcc75e0f0` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:14.439197 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44a5947-212d-4c70-bb7d-1de6ad7f9ea7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5f1432bd4d44aba8f8a279da573b2b3, user_id=34d42062701a4e10a8685652ae431458, role_id=e0ab41d8ae22474688b5361bcc75e0f0)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:14.444799 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44a5947-212d-4c70-bb7d-1de6ad7f9ea7 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:14.463743 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1411/2811] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:25:14 2022] PUT /identity/v3/projects/b5f1432bd4d44aba8f8a279da573b2b3/users/34d42062701a4e10a8685652ae431458/roles/e0ab41d8ae22474688b5361bcc75e0f0 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:25:14.468763 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34b1e53-1a0e-4e42-bd92-9f67b081a29c admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:14.481949 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34b1e53-1a0e-4e42-bd92-9f67b081a29c admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:14.590384 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbc1cbf-39d2-421c-bd58-3d6bcb387880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:14.591271 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbc1cbf-39d2-421c-bd58-3d6bcb387880 admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:14.591732 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbc1cbf-39d2-421c-bd58-3d6bcb387880 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:14.592128 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbc1cbf-39d2-421c-bd58-3d6bcb387880 admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:14.593151 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbc1cbf-39d2-421c-bd58-3d6bcb387880 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:14.596929 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbc1cbf-39d2-421c-bd58-3d6bcb387880 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:14.617357 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1401/2812] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:25:14 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 20:25:14.622089 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44a5947-212d-4c70-bb7d-1de6ad7f9ea7 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:14.637007 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44a5947-212d-4c70-bb7d-1de6ad7f9ea7 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:14.646805 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b27598-3dd5-4d04-8791-7e507b889fdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:14.647962 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b27598-3dd5-4d04-8791-7e507b889fdf admin admin] REQUEST_METHOD: `PUT` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:14.648476 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b27598-3dd5-4d04-8791-7e507b889fdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:14.648947 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b27598-3dd5-4d04-8791-7e507b889fdf admin admin] PATH_INFO: `/v3/projects/b5f1432bd4d44aba8f8a279da573b2b3/users/34d42062701a4e10a8685652ae431458/roles/e0ab41d8ae22474688b5361bcc75e0f0` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:14.651830 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b27598-3dd5-4d04-8791-7e507b889fdf admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5f1432bd4d44aba8f8a279da573b2b3, user_id=34d42062701a4e10a8685652ae431458, role_id=e0ab41d8ae22474688b5361bcc75e0f0)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:14.655305 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b27598-3dd5-4d04-8791-7e507b889fdf admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:14.668982 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1412/2813] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:25:14 2022] PUT /identity/v3/projects/b5f1432bd4d44aba8f8a279da573b2b3/users/34d42062701a4e10a8685652ae431458/roles/e0ab41d8ae22474688b5361bcc75e0f0 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 20:25:14.680806 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd8690a-c491-47a8-9d56-597c59957a3a None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:14.681155 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd8690a-c491-47a8-9d56-597c59957a3a None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:14.681445 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd8690a-c491-47a8-9d56-597c59957a3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:14.717296 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-7fd8690a-c491-47a8-9d56-597c59957a3a None None] MFA Rules not processed for user `34d42062701a4e10a8685652ae431458`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:25:14.718800 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-7fd8690a-c491-47a8-9d56-597c59957a3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:25:14.755068 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:25:14.811710 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1402/2814] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:25:14 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:25:14.828921 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b27598-3dd5-4d04-8791-7e507b889fdf admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:14.832172 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-12b27598-3dd5-4d04-8791-7e507b889fdf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:25:14.917841 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b27598-3dd5-4d04-8791-7e507b889fdf admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:14.922711 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81816675-f0bf-4d45-9541-b8f35c3cb5bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df1bca993bc142908cc7cf85eabd2b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:14.923879 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81816675-f0bf-4d45-9541-b8f35c3cb5bd service cinder] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:14.924158 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81816675-f0bf-4d45-9541-b8f35c3cb5bd service cinder] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:14.924495 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81816675-f0bf-4d45-9541-b8f35c3cb5bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:14.940017 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81816675-f0bf-4d45-9541-b8f35c3cb5bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:14.942756 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81816675-f0bf-4d45-9541-b8f35c3cb5bd service cinder] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:14.961652 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1413/2815] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:25:14 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:25:15.012983 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e035d090-0c39-417a-9ff2-fd4dc520b8cd None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:15.012983 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e035d090-0c39-417a-9ff2-fd4dc520b8cd None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:15.013270 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e035d090-0c39-417a-9ff2-fd4dc520b8cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:15.023018 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81816675-f0bf-4d45-9541-b8f35c3cb5bd service cinder] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:15.024549 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-81816675-f0bf-4d45-9541-b8f35c3cb5bd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:25:15.038190 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81816675-f0bf-4d45-9541-b8f35c3cb5bd service cinder] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:15.039268 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-e035d090-0c39-417a-9ff2-fd4dc520b8cd None None] MFA Rules not processed for user `1a52117df70447dba9bbee35d083e44f`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:25:15.041983 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-e035d090-0c39-417a-9ff2-fd4dc520b8cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:25:15.045127 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b6a922-9308-43c6-a30c-8579043ebda3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df1bca993bc142908cc7cf85eabd2b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:15.046046 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b6a922-9308-43c6-a30c-8579043ebda3 service cinder] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:15.046434 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b6a922-9308-43c6-a30c-8579043ebda3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:15.046796 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b6a922-9308-43c6-a30c-8579043ebda3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:15.117124 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1403/2816] 10.222.0.20 () {58 vars in 1009 bytes} [Tue Sep 6 20:25:15 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:25:15.128520 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b6a922-9308-43c6-a30c-8579043ebda3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:15.130221 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b6a922-9308-43c6-a30c-8579043ebda3 service cinder] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:15.143945 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1414/2817] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:25:15 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 20:25:15.183016 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e035d090-0c39-417a-9ff2-fd4dc520b8cd None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:15.183016 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-e035d090-0c39-417a-9ff2-fd4dc520b8cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:25:15.285638 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e035d090-0c39-417a-9ff2-fd4dc520b8cd None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:15.291766 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92072891-d38b-4d1c-867f-9f9570823ec1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b72516236da47e292d78de201b0552b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:15.292412 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92072891-d38b-4d1c-867f-9f9570823ec1 service swift] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:15.292698 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92072891-d38b-4d1c-867f-9f9570823ec1 service swift] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:15.292972 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92072891-d38b-4d1c-867f-9f9570823ec1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:15.302514 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92072891-d38b-4d1c-867f-9f9570823ec1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:15.304216 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92072891-d38b-4d1c-867f-9f9570823ec1 service swift] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:15.313899 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1404/2818] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Sep 6 20:25:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 20:25:16.005094 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a81e17a-3aed-470d-adf3-97896ec2b402 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:16.005745 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a81e17a-3aed-470d-adf3-97896ec2b402 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:16.005745 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a81e17a-3aed-470d-adf3-97896ec2b402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:16.021502 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:25:16.032056 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-8a81e17a-3aed-470d-adf3-97896ec2b402 None None] MFA Rules not processed for user `1a52117df70447dba9bbee35d083e44f`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:25:16.034759 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-8a81e17a-3aed-470d-adf3-97896ec2b402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:25:16.044515 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1415/2819] 10.222.0.20 () {58 vars in 967 bytes} [Tue Sep 6 20:25:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:25:16.058155 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92072891-d38b-4d1c-867f-9f9570823ec1 service swift] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:16.070749 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92072891-d38b-4d1c-867f-9f9570823ec1 service swift] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:16.076302 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4b27ff-0f63-4b2c-a3b7-1fb11bfd6f32 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b72516236da47e292d78de201b0552b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:16.076952 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4b27ff-0f63-4b2c-a3b7-1fb11bfd6f32 service swift] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:16.077223 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4b27ff-0f63-4b2c-a3b7-1fb11bfd6f32 service swift] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:16.077497 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4b27ff-0f63-4b2c-a3b7-1fb11bfd6f32 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:16.087267 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4b27ff-0f63-4b2c-a3b7-1fb11bfd6f32 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:16.089317 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4b27ff-0f63-4b2c-a3b7-1fb11bfd6f32 service swift] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:16.102960 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1405/2820] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Sep 6 20:25:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 20:25:16.393639 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72571b2b-354d-4258-9f12-1db61615de29 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:16.393944 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72571b2b-354d-4258-9f12-1db61615de29 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:16.395041 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72571b2b-354d-4258-9f12-1db61615de29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:16.420411 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-72571b2b-354d-4258-9f12-1db61615de29 None None] MFA Rules not processed for user `1a52117df70447dba9bbee35d083e44f`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:25:16.421812 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-72571b2b-354d-4258-9f12-1db61615de29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:25:16.430610 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1416/2821] 10.222.0.20 () {58 vars in 1009 bytes} [Tue Sep 6 20:25:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:25:16.445057 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4b27ff-0f63-4b2c-a3b7-1fb11bfd6f32 service swift] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:16.465469 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4b27ff-0f63-4b2c-a3b7-1fb11bfd6f32 service swift] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:16.474873 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e223fb6-0fb2-41a9-8d9a-0af61b42ffda service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b72516236da47e292d78de201b0552b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:16.475809 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e223fb6-0fb2-41a9-8d9a-0af61b42ffda service swift] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:16.476314 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e223fb6-0fb2-41a9-8d9a-0af61b42ffda service swift] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:16.476777 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e223fb6-0fb2-41a9-8d9a-0af61b42ffda service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:16.493191 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e223fb6-0fb2-41a9-8d9a-0af61b42ffda service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:16.496279 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e223fb6-0fb2-41a9-8d9a-0af61b42ffda service swift] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:16.512304 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1406/2822] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Sep 6 20:25:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 20:25:16.977150 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ac89d3-9145-42d8-8241-a4887e38e4e9 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:16.977404 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ac89d3-9145-42d8-8241-a4887e38e4e9 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:16.977774 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ac89d3-9145-42d8-8241-a4887e38e4e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:17.007100 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-48ac89d3-9145-42d8-8241-a4887e38e4e9 None None] MFA Rules not processed for user `1a52117df70447dba9bbee35d083e44f`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:25:17.008989 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-48ac89d3-9145-42d8-8241-a4887e38e4e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:25:17.019782 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1417/2823] 10.222.0.20 () {58 vars in 967 bytes} [Tue Sep 6 20:25:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:25:17.037035 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e223fb6-0fb2-41a9-8d9a-0af61b42ffda service swift] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:17.055938 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e223fb6-0fb2-41a9-8d9a-0af61b42ffda service swift] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:17.063522 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefc5eca-247f-4015-a32c-7a911fae78c5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b72516236da47e292d78de201b0552b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:17.064815 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefc5eca-247f-4015-a32c-7a911fae78c5 service swift] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:17.065314 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefc5eca-247f-4015-a32c-7a911fae78c5 service swift] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:17.065860 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefc5eca-247f-4015-a32c-7a911fae78c5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:17.083627 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefc5eca-247f-4015-a32c-7a911fae78c5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:17.087295 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefc5eca-247f-4015-a32c-7a911fae78c5 service swift] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:17.111421 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1407/2824] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Sep 6 20:25:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 20:25:18.486538 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ac89d3-9145-42d8-8241-a4887e38e4e9 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:18.600109 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ac89d3-9145-42d8-8241-a4887e38e4e9 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:18.612027 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d49aae9-76fd-45c9-bf97-99f4cee7cddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:18.612827 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d49aae9-76fd-45c9-bf97-99f4cee7cddb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:18.613239 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d49aae9-76fd-45c9-bf97-99f4cee7cddb admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:18.613493 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d49aae9-76fd-45c9-bf97-99f4cee7cddb admin admin] PATH_INFO: `/v3/users/c6e731ecdfc244408867e35ef2310fb4` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:18.615003 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d49aae9-76fd-45c9-bf97-99f4cee7cddb admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6e731ecdfc244408867e35ef2310fb4)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:18.617963 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d49aae9-76fd-45c9-bf97-99f4cee7cddb admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:18.713303 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-9d49aae9-76fd-45c9-bf97-99f4cee7cddb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6e731ecdfc244408867e35ef2310fb4'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:18.714475 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-9d49aae9-76fd-45c9-bf97-99f4cee7cddb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6e731ecdfc244408867e35ef2310fb4'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:18.730208 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-9d49aae9-76fd-45c9-bf97-99f4cee7cddb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6e731ecdfc244408867e35ef2310fb4'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:18.731335 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-9d49aae9-76fd-45c9-bf97-99f4cee7cddb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6e731ecdfc244408867e35ef2310fb4'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:18.742513 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-9d49aae9-76fd-45c9-bf97-99f4cee7cddb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6e731ecdfc244408867e35ef2310fb4'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:18.766489 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1418/2825] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:25:18 2022] DELETE /identity/v3/users/c6e731ecdfc244408867e35ef2310fb4 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:25:18.773841 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a07f57-b31f-45a8-8058-bdbe188df507 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:18.774502 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a07f57-b31f-45a8-8058-bdbe188df507 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:18.774502 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a07f57-b31f-45a8-8058-bdbe188df507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:18.804807 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-80a07f57-b31f-45a8-8058-bdbe188df507 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:25:18.806818 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-80a07f57-b31f-45a8-8058-bdbe188df507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:25:18.828639 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=100 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:25:18.912966 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1408/2826] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:25:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:25:18.925295 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d49aae9-76fd-45c9-bf97-99f4cee7cddb admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:18.926862 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-9d49aae9-76fd-45c9-bf97-99f4cee7cddb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:25:18.996457 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d49aae9-76fd-45c9-bf97-99f4cee7cddb admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:19.000720 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2f212f-99e4-4c5e-97f5-34ee69e64fc5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:19.002847 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2f212f-99e4-4c5e-97f5-34ee69e64fc5 service neutron] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:19.002847 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2f212f-99e4-4c5e-97f5-34ee69e64fc5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:19.003291 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2f212f-99e4-4c5e-97f5-34ee69e64fc5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:19.016509 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2f212f-99e4-4c5e-97f5-34ee69e64fc5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:19.018191 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2f212f-99e4-4c5e-97f5-34ee69e64fc5 service neutron] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:19.030116 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1419/2827] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:25:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:25:19.444162 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a07f57-b31f-45a8-8058-bdbe188df507 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:19.446315 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-80a07f57-b31f-45a8-8058-bdbe188df507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:25:19.465097 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a07f57-b31f-45a8-8058-bdbe188df507 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:19.471666 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4cae77-02aa-4b4b-ba72-982f3d5f3eb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:19.472345 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4cae77-02aa-4b4b-ba72-982f3d5f3eb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:19.472989 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4cae77-02aa-4b4b-ba72-982f3d5f3eb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:19.474394 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4cae77-02aa-4b4b-ba72-982f3d5f3eb4 admin admin] PATH_INFO: `/v3/projects/eff5a2e6f99d4210b1580e5ad20868e1` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:19.476098 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4cae77-02aa-4b4b-ba72-982f3d5f3eb4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eff5a2e6f99d4210b1580e5ad20868e1)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:19.478440 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4cae77-02aa-4b4b-ba72-982f3d5f3eb4 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:19.547841 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-6b4cae77-02aa-4b4b-ba72-982f3d5f3eb4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eff5a2e6f99d4210b1580e5ad20868e1', 'request_id': 'req-6b4cae77-02aa-4b4b-ba72-982f3d5f3eb4'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:19.556800 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-6b4cae77-02aa-4b4b-ba72-982f3d5f3eb4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eff5a2e6f99d4210b1580e5ad20868e1', 'request_id': 'req-6b4cae77-02aa-4b4b-ba72-982f3d5f3eb4'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:19.568058 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-6b4cae77-02aa-4b4b-ba72-982f3d5f3eb4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eff5a2e6f99d4210b1580e5ad20868e1', 'request_id': 'req-6b4cae77-02aa-4b4b-ba72-982f3d5f3eb4'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:19.582358 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-6b4cae77-02aa-4b4b-ba72-982f3d5f3eb4 admin admin] The token cache is being invalidate because project eff5a2e6f99d4210b1580e5ad20868e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81771) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:25:19.582676 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-6b4cae77-02aa-4b4b-ba72-982f3d5f3eb4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:19.583361 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-6b4cae77-02aa-4b4b-ba72-982f3d5f3eb4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:19.584951 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1409/2828] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:25:19 2022] DELETE /identity/v3/projects/eff5a2e6f99d4210b1580e5ad20868e1 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 20:25:19.591333 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2f212f-99e4-4c5e-97f5-34ee69e64fc5 service neutron] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:19.609871 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2f212f-99e4-4c5e-97f5-34ee69e64fc5 service neutron] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:19.619605 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:25:19.687696 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4cae77-02aa-4b4b-ba72-982f3d5f3eb4 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:19.690169 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-6b4cae77-02aa-4b4b-ba72-982f3d5f3eb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:25:19.709220 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8b0382-54f4-4806-bf30-77537b76334f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:19.710592 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8b0382-54f4-4806-bf30-77537b76334f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:19.711270 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8b0382-54f4-4806-bf30-77537b76334f admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:19.711814 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8b0382-54f4-4806-bf30-77537b76334f admin admin] PATH_INFO: `/v3/users/43ebba6fee7d4113b805455b1ba9d43d` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:19.714280 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8b0382-54f4-4806-bf30-77537b76334f admin admin] RBAC: Authorizing `identity:delete_user(user_id=43ebba6fee7d4113b805455b1ba9d43d)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:19.717944 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8b0382-54f4-4806-bf30-77537b76334f admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:19.789556 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-2d8b0382-54f4-4806-bf30-77537b76334f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43ebba6fee7d4113b805455b1ba9d43d'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:19.790648 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-2d8b0382-54f4-4806-bf30-77537b76334f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43ebba6fee7d4113b805455b1ba9d43d'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:19.797292 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4cae77-02aa-4b4b-ba72-982f3d5f3eb4 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:19.803640 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b66133-5f8b-419a-8900-b0016b1ec31c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:19.804562 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b66133-5f8b-419a-8900-b0016b1ec31c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:19.804952 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b66133-5f8b-419a-8900-b0016b1ec31c admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:19.805413 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b66133-5f8b-419a-8900-b0016b1ec31c admin admin] PATH_INFO: `/v3/users/1a7236719669467383c242d0f5a91604` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:19.807110 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b66133-5f8b-419a-8900-b0016b1ec31c admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a7236719669467383c242d0f5a91604)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:19.807373 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-2d8b0382-54f4-4806-bf30-77537b76334f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43ebba6fee7d4113b805455b1ba9d43d'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:19.808267 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-2d8b0382-54f4-4806-bf30-77537b76334f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43ebba6fee7d4113b805455b1ba9d43d'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:19.809562 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b66133-5f8b-419a-8900-b0016b1ec31c admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:19.818973 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-2d8b0382-54f4-4806-bf30-77537b76334f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43ebba6fee7d4113b805455b1ba9d43d'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:19.837094 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1420/2829] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:25:19 2022] DELETE /identity/v3/users/43ebba6fee7d4113b805455b1ba9d43d => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:25:19.874375 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:25:19.878934 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-f2b66133-5f8b-419a-8900-b0016b1ec31c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a7236719669467383c242d0f5a91604'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:19.891664 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-f2b66133-5f8b-419a-8900-b0016b1ec31c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a7236719669467383c242d0f5a91604'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:19.892437 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-f2b66133-5f8b-419a-8900-b0016b1ec31c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a7236719669467383c242d0f5a91604'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:19.893134 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-f2b66133-5f8b-419a-8900-b0016b1ec31c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a7236719669467383c242d0f5a91604'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:19.904823 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-f2b66133-5f8b-419a-8900-b0016b1ec31c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a7236719669467383c242d0f5a91604'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:19.934535 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1410/2830] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:25:19 2022] DELETE /identity/v3/users/1a7236719669467383c242d0f5a91604 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:25:19.940268 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cf9828-fd6f-4f81-946f-fd3a5203bc3d None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:19.940594 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cf9828-fd6f-4f81-946f-fd3a5203bc3d None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:19.940818 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cf9828-fd6f-4f81-946f-fd3a5203bc3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:19.973174 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-b4cf9828-fd6f-4f81-946f-fd3a5203bc3d None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:25:19.974525 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-b4cf9828-fd6f-4f81-946f-fd3a5203bc3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:25:20.077039 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1421/2831] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:25:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:25:20.090025 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b66133-5f8b-419a-8900-b0016b1ec31c admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:20.091578 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-f2b66133-5f8b-419a-8900-b0016b1ec31c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:25:20.176179 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b66133-5f8b-419a-8900-b0016b1ec31c admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:20.180898 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed927986-9c9e-4c8d-98db-ca74924c3499 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:20.181642 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed927986-9c9e-4c8d-98db-ca74924c3499 service neutron] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:20.181953 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed927986-9c9e-4c8d-98db-ca74924c3499 service neutron] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:20.182224 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed927986-9c9e-4c8d-98db-ca74924c3499 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:20.193151 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed927986-9c9e-4c8d-98db-ca74924c3499 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:20.195011 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed927986-9c9e-4c8d-98db-ca74924c3499 service neutron] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:20.211245 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1411/2832] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:25:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:25:20.365478 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cf9828-fd6f-4f81-946f-fd3a5203bc3d None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:20.367735 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-b4cf9828-fd6f-4f81-946f-fd3a5203bc3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:25:20.395670 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cf9828-fd6f-4f81-946f-fd3a5203bc3d None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:20.403461 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93286ba5-e7c1-4b5a-95ac-4a02e982c8fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:20.404423 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93286ba5-e7c1-4b5a-95ac-4a02e982c8fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:20.404773 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93286ba5-e7c1-4b5a-95ac-4a02e982c8fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:20.404940 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93286ba5-e7c1-4b5a-95ac-4a02e982c8fd admin admin] PATH_INFO: `/v3/projects/924a36b66a3f4082b35cc4770e2e9cac` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:20.406620 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93286ba5-e7c1-4b5a-95ac-4a02e982c8fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=924a36b66a3f4082b35cc4770e2e9cac)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:20.409519 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93286ba5-e7c1-4b5a-95ac-4a02e982c8fd admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:20.461028 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:25:20.498686 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed927986-9c9e-4c8d-98db-ca74924c3499 service neutron] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:20.500054 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-93286ba5-e7c1-4b5a-95ac-4a02e982c8fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '924a36b66a3f4082b35cc4770e2e9cac', 'request_id': 'req-93286ba5-e7c1-4b5a-95ac-4a02e982c8fd'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:20.500302 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-ed927986-9c9e-4c8d-98db-ca74924c3499 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:25:20.518788 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-93286ba5-e7c1-4b5a-95ac-4a02e982c8fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '924a36b66a3f4082b35cc4770e2e9cac', 'request_id': 'req-93286ba5-e7c1-4b5a-95ac-4a02e982c8fd'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:20.525022 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed927986-9c9e-4c8d-98db-ca74924c3499 service neutron] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:20.535489 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-93286ba5-e7c1-4b5a-95ac-4a02e982c8fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '924a36b66a3f4082b35cc4770e2e9cac', 'request_id': 'req-93286ba5-e7c1-4b5a-95ac-4a02e982c8fd'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:20.559116 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-93286ba5-e7c1-4b5a-95ac-4a02e982c8fd admin admin] The token cache is being invalidate because project 924a36b66a3f4082b35cc4770e2e9cac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81772) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:25:20.559554 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-93286ba5-e7c1-4b5a-95ac-4a02e982c8fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:20.560424 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-93286ba5-e7c1-4b5a-95ac-4a02e982c8fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:20.562216 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1422/2833] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:25:20 2022] DELETE /identity/v3/projects/924a36b66a3f4082b35cc4770e2e9cac => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:25:20.567482 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93286ba5-e7c1-4b5a-95ac-4a02e982c8fd admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:20.569805 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-93286ba5-e7c1-4b5a-95ac-4a02e982c8fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:25:20.614900 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3baaa3-3585-4d91-ad9d-d1433e669f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:20.615978 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3baaa3-3585-4d91-ad9d-d1433e669f58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:20.616464 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3baaa3-3585-4d91-ad9d-d1433e669f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:20.616937 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3baaa3-3585-4d91-ad9d-d1433e669f58 admin admin] PATH_INFO: `/v3/projects/7709cf0cbdb5418d92f698a66780e61f` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:20.619230 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3baaa3-3585-4d91-ad9d-d1433e669f58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7709cf0cbdb5418d92f698a66780e61f)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:20.622910 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3baaa3-3585-4d91-ad9d-d1433e669f58 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:20.675056 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93286ba5-e7c1-4b5a-95ac-4a02e982c8fd admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:20.682687 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547ea285-b9c8-4f49-aa01-07b8455edf4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:20.683739 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547ea285-b9c8-4f49-aa01-07b8455edf4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:20.684331 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547ea285-b9c8-4f49-aa01-07b8455edf4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:20.684764 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547ea285-b9c8-4f49-aa01-07b8455edf4d admin admin] PATH_INFO: `/v3/users/34d42062701a4e10a8685652ae431458` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:20.686505 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547ea285-b9c8-4f49-aa01-07b8455edf4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=34d42062701a4e10a8685652ae431458)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:20.690384 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547ea285-b9c8-4f49-aa01-07b8455edf4d admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:20.721458 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-1a3baaa3-3585-4d91-ad9d-d1433e669f58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7709cf0cbdb5418d92f698a66780e61f', 'request_id': 'req-1a3baaa3-3585-4d91-ad9d-d1433e669f58'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:20.732486 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-1a3baaa3-3585-4d91-ad9d-d1433e669f58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7709cf0cbdb5418d92f698a66780e61f', 'request_id': 'req-1a3baaa3-3585-4d91-ad9d-d1433e669f58'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:20.744556 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-1a3baaa3-3585-4d91-ad9d-d1433e669f58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7709cf0cbdb5418d92f698a66780e61f', 'request_id': 'req-1a3baaa3-3585-4d91-ad9d-d1433e669f58'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:20.753357 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:25:20.767176 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-1a3baaa3-3585-4d91-ad9d-d1433e669f58 admin admin] The token cache is being invalidate because project 7709cf0cbdb5418d92f698a66780e61f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81771) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:25:20.768373 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-1a3baaa3-3585-4d91-ad9d-d1433e669f58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:20.769308 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-547ea285-b9c8-4f49-aa01-07b8455edf4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34d42062701a4e10a8685652ae431458'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:20.769793 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-1a3baaa3-3585-4d91-ad9d-d1433e669f58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:20.770687 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-547ea285-b9c8-4f49-aa01-07b8455edf4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34d42062701a4e10a8685652ae431458'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:20.785678 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1412/2834] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:25:20 2022] DELETE /identity/v3/projects/7709cf0cbdb5418d92f698a66780e61f => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:25:20.792082 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-547ea285-b9c8-4f49-aa01-07b8455edf4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34d42062701a4e10a8685652ae431458'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:20.793105 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-547ea285-b9c8-4f49-aa01-07b8455edf4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34d42062701a4e10a8685652ae431458'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:20.805751 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-547ea285-b9c8-4f49-aa01-07b8455edf4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34d42062701a4e10a8685652ae431458'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:20.823508 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1423/2835] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:25:20 2022] DELETE /identity/v3/users/34d42062701a4e10a8685652ae431458 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 20:25:21.291778 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3baaa3-3585-4d91-ad9d-d1433e669f58 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:21.294223 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-1a3baaa3-3585-4d91-ad9d-d1433e669f58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:25:21.414669 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3baaa3-3585-4d91-ad9d-d1433e669f58 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:21.493646 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dc3bad-0077-4e84-9559-a9287d095d6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:21.494612 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dc3bad-0077-4e84-9559-a9287d095d6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:21.494945 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dc3bad-0077-4e84-9559-a9287d095d6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:21.495279 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dc3bad-0077-4e84-9559-a9287d095d6f admin admin] PATH_INFO: `/v3/projects/b5f1432bd4d44aba8f8a279da573b2b3` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:21.496852 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dc3bad-0077-4e84-9559-a9287d095d6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5f1432bd4d44aba8f8a279da573b2b3)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:21.499186 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dc3bad-0077-4e84-9559-a9287d095d6f admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:21.559841 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-04dc3bad-0077-4e84-9559-a9287d095d6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5f1432bd4d44aba8f8a279da573b2b3', 'request_id': 'req-04dc3bad-0077-4e84-9559-a9287d095d6f'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:21.568808 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-04dc3bad-0077-4e84-9559-a9287d095d6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5f1432bd4d44aba8f8a279da573b2b3', 'request_id': 'req-04dc3bad-0077-4e84-9559-a9287d095d6f'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:21.578995 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-04dc3bad-0077-4e84-9559-a9287d095d6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5f1432bd4d44aba8f8a279da573b2b3', 'request_id': 'req-04dc3bad-0077-4e84-9559-a9287d095d6f'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:21.593283 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-04dc3bad-0077-4e84-9559-a9287d095d6f admin admin] The token cache is being invalidate because project b5f1432bd4d44aba8f8a279da573b2b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81771) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:25:21.593656 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-04dc3bad-0077-4e84-9559-a9287d095d6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:21.594627 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-04dc3bad-0077-4e84-9559-a9287d095d6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:21.596335 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1413/2836] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:25:21 2022] DELETE /identity/v3/projects/b5f1432bd4d44aba8f8a279da573b2b3 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:25:21.603069 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1ef259-1768-4d0e-9f4a-ba799077d77e None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:21.603411 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1ef259-1768-4d0e-9f4a-ba799077d77e None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:21.603738 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1ef259-1768-4d0e-9f4a-ba799077d77e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:21.627347 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-4b1ef259-1768-4d0e-9f4a-ba799077d77e None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:25:21.628605 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-4b1ef259-1768-4d0e-9f4a-ba799077d77e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:25:21.665721 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:25:21.723879 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1424/2837] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:25:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:25:21.738550 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1ba2dc-b036-4670-983e-2b28e2bcbc5a None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:21.739002 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1ba2dc-b036-4670-983e-2b28e2bcbc5a None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:21.739443 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1ba2dc-b036-4670-983e-2b28e2bcbc5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:21.771682 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-0f1ba2dc-b036-4670-983e-2b28e2bcbc5a None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:25:21.773971 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-0f1ba2dc-b036-4670-983e-2b28e2bcbc5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:25:21.790194 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1414/2838] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:25:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:25:21.796685 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1ef259-1768-4d0e-9f4a-ba799077d77e None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:21.810613 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1ef259-1768-4d0e-9f4a-ba799077d77e None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:21.817371 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0611f56-4e66-41e7-99ba-7fd34b9bc278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:21.818119 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0611f56-4e66-41e7-99ba-7fd34b9bc278 admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:21.818447 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0611f56-4e66-41e7-99ba-7fd34b9bc278 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:21.818793 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0611f56-4e66-41e7-99ba-7fd34b9bc278 admin admin] PATH_INFO: `/v3/domains` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:21.819621 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0611f56-4e66-41e7-99ba-7fd34b9bc278 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:25:21.819974 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0611f56-4e66-41e7-99ba-7fd34b9bc278 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:21.822243 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0611f56-4e66-41e7-99ba-7fd34b9bc278 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:21.850589 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1425/2839] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 6 20:25:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:25:21.856181 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1ba2dc-b036-4670-983e-2b28e2bcbc5a None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:21.868852 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1ba2dc-b036-4670-983e-2b28e2bcbc5a None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:21.875274 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab31cc85-0ae4-4b0a-8be7-247a8ff2ab01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:21.876566 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab31cc85-0ae4-4b0a-8be7-247a8ff2ab01 admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:21.876880 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab31cc85-0ae4-4b0a-8be7-247a8ff2ab01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:21.877327 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab31cc85-0ae4-4b0a-8be7-247a8ff2ab01 admin admin] PATH_INFO: `/v3/projects` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:21.878383 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab31cc85-0ae4-4b0a-8be7-247a8ff2ab01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:21.881111 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab31cc85-0ae4-4b0a-8be7-247a8ff2ab01 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:21.922876 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1415/2840] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:25:21 2022] POST /identity/v3/projects => generated 387 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:25:21.928591 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0611f56-4e66-41e7-99ba-7fd34b9bc278 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:21.941077 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0611f56-4e66-41e7-99ba-7fd34b9bc278 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:22.030712 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4b7e5e-20b2-4f62-992a-258410679aa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:22.031804 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4b7e5e-20b2-4f62-992a-258410679aa4 admin admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:22.032298 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4b7e5e-20b2-4f62-992a-258410679aa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:22.033421 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4b7e5e-20b2-4f62-992a-258410679aa4 admin admin] PATH_INFO: `/v3/users` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:22.034569 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4b7e5e-20b2-4f62-992a-258410679aa4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:22.038312 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4b7e5e-20b2-4f62-992a-258410679aa4 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:22.077667 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1426/2841] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:25:21 2022] POST /identity/v3/users => generated 294 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:25:22.084629 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab31cc85-0ae4-4b0a-8be7-247a8ff2ab01 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:22.095580 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:DELETE=8,keystone:SELECT=90,keystone:INSERT=2 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:25:22.105902 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab31cc85-0ae4-4b0a-8be7-247a8ff2ab01 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:22.116408 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdd0b38-a511-4757-9280-288a77e5c345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:22.117533 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdd0b38-a511-4757-9280-288a77e5c345 admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:22.118121 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdd0b38-a511-4757-9280-288a77e5c345 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:22.118711 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdd0b38-a511-4757-9280-288a77e5c345 admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:22.120159 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdd0b38-a511-4757-9280-288a77e5c345 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:22.123595 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdd0b38-a511-4757-9280-288a77e5c345 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:22.147670 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1416/2842] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:25:22 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:25:22.154899 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4b7e5e-20b2-4f62-992a-258410679aa4 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:22.173864 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4b7e5e-20b2-4f62-992a-258410679aa4 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:22.182667 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7375a1f-6904-4677-8752-c87533eff472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:22.183711 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7375a1f-6904-4677-8752-c87533eff472 admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:22.184185 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7375a1f-6904-4677-8752-c87533eff472 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:22.184673 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7375a1f-6904-4677-8752-c87533eff472 admin admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:22.185920 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7375a1f-6904-4677-8752-c87533eff472 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:22.189641 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7375a1f-6904-4677-8752-c87533eff472 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:22.211255 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1427/2843] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:25:22 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:25:22.217218 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdd0b38-a511-4757-9280-288a77e5c345 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:22.237534 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdd0b38-a511-4757-9280-288a77e5c345 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:22.247559 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ba789c-74bb-4a38-aa8d-2a341c2a184a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:22.248136 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ba789c-74bb-4a38-aa8d-2a341c2a184a admin admin] REQUEST_METHOD: `PUT` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:22.248621 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ba789c-74bb-4a38-aa8d-2a341c2a184a admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:22.249110 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ba789c-74bb-4a38-aa8d-2a341c2a184a admin admin] PATH_INFO: `/v3/projects/1a324208111e4bad8357c55b1e382d96/users/b7ca079aa0ea473bbb0824ea433f9ad7/roles/4eea39e24b06431aa357053fe7907a3d` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:22.272345 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ba789c-74bb-4a38-aa8d-2a341c2a184a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a324208111e4bad8357c55b1e382d96, user_id=b7ca079aa0ea473bbb0824ea433f9ad7, role_id=4eea39e24b06431aa357053fe7907a3d)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:22.279899 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ba789c-74bb-4a38-aa8d-2a341c2a184a admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:22.302479 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1417/2844] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:25:22 2022] PUT /identity/v3/projects/1a324208111e4bad8357c55b1e382d96/users/b7ca079aa0ea473bbb0824ea433f9ad7/roles/4eea39e24b06431aa357053fe7907a3d => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:25:22.413935 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257f5c0e-1cee-4c3c-85d7-7ee5e307ab16 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:22.414437 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257f5c0e-1cee-4c3c-85d7-7ee5e307ab16 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:22.414935 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257f5c0e-1cee-4c3c-85d7-7ee5e307ab16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:22.467145 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-257f5c0e-1cee-4c3c-85d7-7ee5e307ab16 None None] MFA Rules not processed for user `b7ca079aa0ea473bbb0824ea433f9ad7`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:25:22.469328 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-257f5c0e-1cee-4c3c-85d7-7ee5e307ab16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:25:22.600221 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1428/2845] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:25:22 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:25:22.606564 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ba789c-74bb-4a38-aa8d-2a341c2a184a admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:22.708209 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ba789c-74bb-4a38-aa8d-2a341c2a184a admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:22.713997 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4976f4-e715-4c00-816c-5f7419639c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:22.715078 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4976f4-e715-4c00-816c-5f7419639c4b admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:22.715293 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4976f4-e715-4c00-816c-5f7419639c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:22.715649 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4976f4-e715-4c00-816c-5f7419639c4b admin admin] PATH_INFO: `/v3/projects` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:22.716367 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4976f4-e715-4c00-816c-5f7419639c4b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:22.718537 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4976f4-e715-4c00-816c-5f7419639c4b admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:22.757530 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1418/2846] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:25:22 2022] POST /identity/v3/projects => generated 385 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:25:22.763527 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257f5c0e-1cee-4c3c-85d7-7ee5e307ab16 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:22.778191 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257f5c0e-1cee-4c3c-85d7-7ee5e307ab16 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:22.818760 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:25:22.886377 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838d5f85-6572-4a1b-b4e9-ef30a3bfd76b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:22.887431 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838d5f85-6572-4a1b-b4e9-ef30a3bfd76b admin admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:22.888048 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838d5f85-6572-4a1b-b4e9-ef30a3bfd76b admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:22.888447 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838d5f85-6572-4a1b-b4e9-ef30a3bfd76b admin admin] PATH_INFO: `/v3/users` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:22.889832 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838d5f85-6572-4a1b-b4e9-ef30a3bfd76b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:22.893788 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838d5f85-6572-4a1b-b4e9-ef30a3bfd76b admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:22.927768 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1429/2847] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:25:22 2022] POST /identity/v3/users => generated 299 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 20:25:22.934599 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4976f4-e715-4c00-816c-5f7419639c4b admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:22.955032 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4976f4-e715-4c00-816c-5f7419639c4b admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:22.963510 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cfebcf-26a2-4c82-ad7c-41b738f12ab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:22.964448 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cfebcf-26a2-4c82-ad7c-41b738f12ab9 admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:22.964935 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cfebcf-26a2-4c82-ad7c-41b738f12ab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:22.965431 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cfebcf-26a2-4c82-ad7c-41b738f12ab9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:22.966688 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cfebcf-26a2-4c82-ad7c-41b738f12ab9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:22.969813 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cfebcf-26a2-4c82-ad7c-41b738f12ab9 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:22.991067 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1419/2848] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:25:22 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:25:22.997208 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838d5f85-6572-4a1b-b4e9-ef30a3bfd76b admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:23.010475 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838d5f85-6572-4a1b-b4e9-ef30a3bfd76b admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:23.016302 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a3081a-675b-4ff7-9c8f-5f7f23b3c628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:23.017250 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a3081a-675b-4ff7-9c8f-5f7f23b3c628 admin admin] REQUEST_METHOD: `PUT` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:23.018402 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a3081a-675b-4ff7-9c8f-5f7f23b3c628 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:23.018402 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a3081a-675b-4ff7-9c8f-5f7f23b3c628 admin admin] PATH_INFO: `/v3/projects/cf6c5e588a724591939834bd23eb5791/users/7cf4d7a79a4f40c196a1b75950e991c4/roles/e0ab41d8ae22474688b5361bcc75e0f0` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:23.037816 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a3081a-675b-4ff7-9c8f-5f7f23b3c628 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf6c5e588a724591939834bd23eb5791, user_id=7cf4d7a79a4f40c196a1b75950e991c4, role_id=e0ab41d8ae22474688b5361bcc75e0f0)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:23.041099 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a3081a-675b-4ff7-9c8f-5f7f23b3c628 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:23.061262 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1430/2849] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:25:22 2022] PUT /identity/v3/projects/cf6c5e588a724591939834bd23eb5791/users/7cf4d7a79a4f40c196a1b75950e991c4/roles/e0ab41d8ae22474688b5361bcc75e0f0 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 20:25:23.068094 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cfebcf-26a2-4c82-ad7c-41b738f12ab9 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:23.084898 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cfebcf-26a2-4c82-ad7c-41b738f12ab9 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:23.175044 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83566f33-e6fb-49fe-bbb8-2f9e1427494f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:23.175698 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83566f33-e6fb-49fe-bbb8-2f9e1427494f admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:23.176076 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83566f33-e6fb-49fe-bbb8-2f9e1427494f admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:23.176354 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83566f33-e6fb-49fe-bbb8-2f9e1427494f admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:23.177053 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83566f33-e6fb-49fe-bbb8-2f9e1427494f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:23.178804 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83566f33-e6fb-49fe-bbb8-2f9e1427494f admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:23.193352 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1420/2850] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:25:23 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:25:23.199276 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a3081a-675b-4ff7-9c8f-5f7f23b3c628 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:23.216824 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a3081a-675b-4ff7-9c8f-5f7f23b3c628 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:23.222477 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab2fb87-f0f8-4087-80e6-9a2d6607dd3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:23.223278 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab2fb87-f0f8-4087-80e6-9a2d6607dd3a admin admin] REQUEST_METHOD: `PUT` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:23.223562 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab2fb87-f0f8-4087-80e6-9a2d6607dd3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:23.223848 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab2fb87-f0f8-4087-80e6-9a2d6607dd3a admin admin] PATH_INFO: `/v3/projects/cf6c5e588a724591939834bd23eb5791/users/7cf4d7a79a4f40c196a1b75950e991c4/roles/e0ab41d8ae22474688b5361bcc75e0f0` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:23.226469 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab2fb87-f0f8-4087-80e6-9a2d6607dd3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf6c5e588a724591939834bd23eb5791, user_id=7cf4d7a79a4f40c196a1b75950e991c4, role_id=e0ab41d8ae22474688b5361bcc75e0f0)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:23.229438 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab2fb87-f0f8-4087-80e6-9a2d6607dd3a admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:23.249230 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1431/2851] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:25:23 2022] PUT /identity/v3/projects/cf6c5e588a724591939834bd23eb5791/users/7cf4d7a79a4f40c196a1b75950e991c4/roles/e0ab41d8ae22474688b5361bcc75e0f0 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:25:23.265291 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85e22fd-f5cd-4929-903f-35d74864889a None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:23.265801 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85e22fd-f5cd-4929-903f-35d74864889a None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:23.266224 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85e22fd-f5cd-4929-903f-35d74864889a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:23.308935 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-d85e22fd-f5cd-4929-903f-35d74864889a None None] MFA Rules not processed for user `7cf4d7a79a4f40c196a1b75950e991c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:25:23.310756 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-d85e22fd-f5cd-4929-903f-35d74864889a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:25:23.352357 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:25:23.427715 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1421/2852] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:25:23 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:25:23.447249 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab2fb87-f0f8-4087-80e6-9a2d6607dd3a admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:23.450002 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-aab2fb87-f0f8-4087-80e6-9a2d6607dd3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:25:23.544781 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab2fb87-f0f8-4087-80e6-9a2d6607dd3a admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:23.550487 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e87f51c-db8d-42a7-ae07-87ca0c4d7f74 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df1bca993bc142908cc7cf85eabd2b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:23.551596 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e87f51c-db8d-42a7-ae07-87ca0c4d7f74 service cinder] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:23.552039 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e87f51c-db8d-42a7-ae07-87ca0c4d7f74 service cinder] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:23.552541 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e87f51c-db8d-42a7-ae07-87ca0c4d7f74 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:23.570551 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e87f51c-db8d-42a7-ae07-87ca0c4d7f74 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:23.573406 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e87f51c-db8d-42a7-ae07-87ca0c4d7f74 service cinder] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:23.592656 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1432/2853] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:25:23 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:25:24.529040 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85e22fd-f5cd-4929-903f-35d74864889a None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:24.632777 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85e22fd-f5cd-4929-903f-35d74864889a None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:24.642542 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b01d237-55a4-4ba5-8545-6801a2f23e8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:24.644119 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b01d237-55a4-4ba5-8545-6801a2f23e8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:24.644594 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b01d237-55a4-4ba5-8545-6801a2f23e8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:24.645071 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b01d237-55a4-4ba5-8545-6801a2f23e8e admin admin] PATH_INFO: `/v3/users/b7ca079aa0ea473bbb0824ea433f9ad7` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:24.647560 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b01d237-55a4-4ba5-8545-6801a2f23e8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7ca079aa0ea473bbb0824ea433f9ad7)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:24.651490 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b01d237-55a4-4ba5-8545-6801a2f23e8e admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:24.739461 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-9b01d237-55a4-4ba5-8545-6801a2f23e8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7ca079aa0ea473bbb0824ea433f9ad7'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:24.750808 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-9b01d237-55a4-4ba5-8545-6801a2f23e8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7ca079aa0ea473bbb0824ea433f9ad7'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:24.751598 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-9b01d237-55a4-4ba5-8545-6801a2f23e8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7ca079aa0ea473bbb0824ea433f9ad7'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:24.752274 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-9b01d237-55a4-4ba5-8545-6801a2f23e8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7ca079aa0ea473bbb0824ea433f9ad7'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:24.762673 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-9b01d237-55a4-4ba5-8545-6801a2f23e8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7ca079aa0ea473bbb0824ea433f9ad7'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:24.797215 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1422/2854] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:25:24 2022] DELETE /identity/v3/users/b7ca079aa0ea473bbb0824ea433f9ad7 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 20:25:24.804507 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c093ef-473b-4194-9355-8aed489ff3f2 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:24.805417 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c093ef-473b-4194-9355-8aed489ff3f2 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:24.805417 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c093ef-473b-4194-9355-8aed489ff3f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:24.828343 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-e1c093ef-473b-4194-9355-8aed489ff3f2 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:25:24.830459 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-e1c093ef-473b-4194-9355-8aed489ff3f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:25:24.913684 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1433/2855] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:25:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:25:24.929736 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b01d237-55a4-4ba5-8545-6801a2f23e8e admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:24.932178 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-9b01d237-55a4-4ba5-8545-6801a2f23e8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:25:24.992877 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:25:25.012126 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b01d237-55a4-4ba5-8545-6801a2f23e8e admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:25.016710 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27567dc-4afa-4a68-bf78-d1515b3493d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:25.017412 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27567dc-4afa-4a68-bf78-d1515b3493d5 service neutron] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:25.017737 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27567dc-4afa-4a68-bf78-d1515b3493d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:25.018022 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27567dc-4afa-4a68-bf78-d1515b3493d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:25.034757 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27567dc-4afa-4a68-bf78-d1515b3493d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:25.037649 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27567dc-4afa-4a68-bf78-d1515b3493d5 service neutron] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:25.058371 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1423/2856] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:25:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:25:25.429271 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c093ef-473b-4194-9355-8aed489ff3f2 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:25.431298 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-e1c093ef-473b-4194-9355-8aed489ff3f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:25:25.446299 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c093ef-473b-4194-9355-8aed489ff3f2 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:25.452603 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b5119d-d48f-4117-8652-b2650ce14c8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:25.453274 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b5119d-d48f-4117-8652-b2650ce14c8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:25.453555 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b5119d-d48f-4117-8652-b2650ce14c8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:25.453879 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b5119d-d48f-4117-8652-b2650ce14c8b admin admin] PATH_INFO: `/v3/projects/1a324208111e4bad8357c55b1e382d96` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:25.455883 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b5119d-d48f-4117-8652-b2650ce14c8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a324208111e4bad8357c55b1e382d96)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:25.457920 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b5119d-d48f-4117-8652-b2650ce14c8b admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:25.484081 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:25:25.528928 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-41b5119d-d48f-4117-8652-b2650ce14c8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a324208111e4bad8357c55b1e382d96', 'request_id': 'req-41b5119d-d48f-4117-8652-b2650ce14c8b'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:25.539280 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-41b5119d-d48f-4117-8652-b2650ce14c8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a324208111e4bad8357c55b1e382d96', 'request_id': 'req-41b5119d-d48f-4117-8652-b2650ce14c8b'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:25.547485 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-41b5119d-d48f-4117-8652-b2650ce14c8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a324208111e4bad8357c55b1e382d96', 'request_id': 'req-41b5119d-d48f-4117-8652-b2650ce14c8b'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:25.561911 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-41b5119d-d48f-4117-8652-b2650ce14c8b admin admin] The token cache is being invalidate because project 1a324208111e4bad8357c55b1e382d96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81772) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:25:25.562312 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-41b5119d-d48f-4117-8652-b2650ce14c8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:25.563257 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-41b5119d-d48f-4117-8652-b2650ce14c8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:25.565015 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1434/2857] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:25:25 2022] DELETE /identity/v3/projects/1a324208111e4bad8357c55b1e382d96 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:25:25.569778 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27567dc-4afa-4a68-bf78-d1515b3493d5 service neutron] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:25.581228 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27567dc-4afa-4a68-bf78-d1515b3493d5 service neutron] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:25.666371 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02940f7a-0b52-44b2-9109-af140e74c209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:25.667042 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02940f7a-0b52-44b2-9109-af140e74c209 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:25.667328 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02940f7a-0b52-44b2-9109-af140e74c209 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:25.667710 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02940f7a-0b52-44b2-9109-af140e74c209 admin admin] PATH_INFO: `/v3/users/7cf4d7a79a4f40c196a1b75950e991c4` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:25.669115 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02940f7a-0b52-44b2-9109-af140e74c209 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cf4d7a79a4f40c196a1b75950e991c4)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:25.671686 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02940f7a-0b52-44b2-9109-af140e74c209 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:25.734249 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-02940f7a-0b52-44b2-9109-af140e74c209 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cf4d7a79a4f40c196a1b75950e991c4'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:25.744227 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-02940f7a-0b52-44b2-9109-af140e74c209 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cf4d7a79a4f40c196a1b75950e991c4'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:25.744893 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-02940f7a-0b52-44b2-9109-af140e74c209 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cf4d7a79a4f40c196a1b75950e991c4'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:25.745846 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-02940f7a-0b52-44b2-9109-af140e74c209 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cf4d7a79a4f40c196a1b75950e991c4'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:25.757069 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-02940f7a-0b52-44b2-9109-af140e74c209 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cf4d7a79a4f40c196a1b75950e991c4'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:25.787323 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1424/2858] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:25:25 2022] DELETE /identity/v3/users/7cf4d7a79a4f40c196a1b75950e991c4 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:25:26.167705 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b5119d-d48f-4117-8652-b2650ce14c8b admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:26.169565 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-41b5119d-d48f-4117-8652-b2650ce14c8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:25:26.192707 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b5119d-d48f-4117-8652-b2650ce14c8b admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:26.295946 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780b1327-444f-4d54-81a6-c96e3703bde7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:26.296620 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780b1327-444f-4d54-81a6-c96e3703bde7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:26.296933 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780b1327-444f-4d54-81a6-c96e3703bde7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:26.297221 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780b1327-444f-4d54-81a6-c96e3703bde7 admin admin] PATH_INFO: `/v3/projects/cf6c5e588a724591939834bd23eb5791` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:26.299263 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780b1327-444f-4d54-81a6-c96e3703bde7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf6c5e588a724591939834bd23eb5791)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:26.301393 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780b1327-444f-4d54-81a6-c96e3703bde7 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:26.362190 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-780b1327-444f-4d54-81a6-c96e3703bde7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf6c5e588a724591939834bd23eb5791', 'request_id': 'req-780b1327-444f-4d54-81a6-c96e3703bde7'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:26.372040 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-780b1327-444f-4d54-81a6-c96e3703bde7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf6c5e588a724591939834bd23eb5791', 'request_id': 'req-780b1327-444f-4d54-81a6-c96e3703bde7'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:26.380541 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-780b1327-444f-4d54-81a6-c96e3703bde7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf6c5e588a724591939834bd23eb5791', 'request_id': 'req-780b1327-444f-4d54-81a6-c96e3703bde7'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:26.397227 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-780b1327-444f-4d54-81a6-c96e3703bde7 admin admin] The token cache is being invalidate because project cf6c5e588a724591939834bd23eb5791 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81772) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:25:26.397776 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-780b1327-444f-4d54-81a6-c96e3703bde7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:26.398706 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-780b1327-444f-4d54-81a6-c96e3703bde7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:26.401084 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1435/2859] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:25:26 2022] DELETE /identity/v3/projects/cf6c5e588a724591939834bd23eb5791 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:25:26.406903 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd87dea-1d1c-4648-9755-fa965e796a9c None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:26.407203 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd87dea-1d1c-4648-9755-fa965e796a9c None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:26.407524 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd87dea-1d1c-4648-9755-fa965e796a9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:26.430924 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-0fd87dea-1d1c-4648-9755-fa965e796a9c None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:25:26.432084 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-0fd87dea-1d1c-4648-9755-fa965e796a9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:25:26.531860 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1425/2860] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:25:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:25:26.542537 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee13d7f-fc4e-41bf-942a-0af56ca5bce2 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:26.542815 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee13d7f-fc4e-41bf-942a-0af56ca5bce2 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:26.543244 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee13d7f-fc4e-41bf-942a-0af56ca5bce2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:26.570044 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-eee13d7f-fc4e-41bf-942a-0af56ca5bce2 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:25:26.571470 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-eee13d7f-fc4e-41bf-942a-0af56ca5bce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:25:26.580752 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1436/2861] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:25:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:25:26.585974 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd87dea-1d1c-4648-9755-fa965e796a9c None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:26.593723 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:25:26.599649 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd87dea-1d1c-4648-9755-fa965e796a9c None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:26.608412 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da7d2a3-2714-4450-84b0-62096a51982a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:26.609080 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da7d2a3-2714-4450-84b0-62096a51982a admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:26.609375 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da7d2a3-2714-4450-84b0-62096a51982a admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:26.609718 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da7d2a3-2714-4450-84b0-62096a51982a admin admin] PATH_INFO: `/v3/domains` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:26.610447 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da7d2a3-2714-4450-84b0-62096a51982a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81771) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:25:26.610804 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da7d2a3-2714-4450-84b0-62096a51982a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:26.612473 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da7d2a3-2714-4450-84b0-62096a51982a admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:26.629732 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1426/2862] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 6 20:25:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 20:25:26.636939 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee13d7f-fc4e-41bf-942a-0af56ca5bce2 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:26.649425 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee13d7f-fc4e-41bf-942a-0af56ca5bce2 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:26.655404 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4a15d3-7988-4ef2-be17-ff9921d8984a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:26.656083 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4a15d3-7988-4ef2-be17-ff9921d8984a admin admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:26.656400 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4a15d3-7988-4ef2-be17-ff9921d8984a admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:26.656716 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4a15d3-7988-4ef2-be17-ff9921d8984a admin admin] PATH_INFO: `/v3/projects` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:26.658151 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4a15d3-7988-4ef2-be17-ff9921d8984a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:26.660190 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4a15d3-7988-4ef2-be17-ff9921d8984a admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:26.700267 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1437/2863] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:25:26 2022] POST /identity/v3/projects => generated 381 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:25:26.705085 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da7d2a3-2714-4450-84b0-62096a51982a admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:26.716431 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da7d2a3-2714-4450-84b0-62096a51982a admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:26.801317 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2072a203-54e4-41ab-b61c-72536645c072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:26.802368 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2072a203-54e4-41ab-b61c-72536645c072 admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:26.802850 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2072a203-54e4-41ab-b61c-72536645c072 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:26.803344 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2072a203-54e4-41ab-b61c-72536645c072 admin admin] PATH_INFO: `/v3/users` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:26.804542 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2072a203-54e4-41ab-b61c-72536645c072 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:26.811624 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2072a203-54e4-41ab-b61c-72536645c072 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:26.844559 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1427/2864] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:25:26 2022] POST /identity/v3/users => generated 291 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 20:25:26.849330 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4a15d3-7988-4ef2-be17-ff9921d8984a admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:26.864020 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4a15d3-7988-4ef2-be17-ff9921d8984a admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:26.870257 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d0e00f-707c-4b68-bcb1-ee0c32e1c2fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:26.871058 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d0e00f-707c-4b68-bcb1-ee0c32e1c2fb admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:26.871535 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d0e00f-707c-4b68-bcb1-ee0c32e1c2fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:26.871832 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d0e00f-707c-4b68-bcb1-ee0c32e1c2fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:26.872511 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d0e00f-707c-4b68-bcb1-ee0c32e1c2fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:26.875429 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d0e00f-707c-4b68-bcb1-ee0c32e1c2fb admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:26.877389 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:25:26.891702 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1438/2865] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:25:26 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:25:26.897229 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2072a203-54e4-41ab-b61c-72536645c072 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:26.913712 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2072a203-54e4-41ab-b61c-72536645c072 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:26.922074 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5ab052-2e68-486d-a96e-70c5fc8865f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:26.923630 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5ab052-2e68-486d-a96e-70c5fc8865f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:26.924123 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5ab052-2e68-486d-a96e-70c5fc8865f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:26.924616 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5ab052-2e68-486d-a96e-70c5fc8865f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:26.925877 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5ab052-2e68-486d-a96e-70c5fc8865f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:26.929029 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5ab052-2e68-486d-a96e-70c5fc8865f9 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:26.952759 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1428/2866] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:25:26 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 20:25:26.959901 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d0e00f-707c-4b68-bcb1-ee0c32e1c2fb admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:26.979101 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d0e00f-707c-4b68-bcb1-ee0c32e1c2fb admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:26.987872 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c7d68a-02f4-4e7a-a1d7-eeb0ae9bdb6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:26.989078 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c7d68a-02f4-4e7a-a1d7-eeb0ae9bdb6e admin admin] REQUEST_METHOD: `PUT` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:26.989591 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c7d68a-02f4-4e7a-a1d7-eeb0ae9bdb6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:26.990135 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c7d68a-02f4-4e7a-a1d7-eeb0ae9bdb6e admin admin] PATH_INFO: `/v3/projects/faf1ef13b2424728ba6f241bf8e65325/users/a4fd86cd67d549e888a01c1e735bc1e5/roles/4eea39e24b06431aa357053fe7907a3d` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:27.010457 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c7d68a-02f4-4e7a-a1d7-eeb0ae9bdb6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=faf1ef13b2424728ba6f241bf8e65325, user_id=a4fd86cd67d549e888a01c1e735bc1e5, role_id=4eea39e24b06431aa357053fe7907a3d)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:27.013240 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c7d68a-02f4-4e7a-a1d7-eeb0ae9bdb6e admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:27.029705 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1439/2867] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:25:26 2022] PUT /identity/v3/projects/faf1ef13b2424728ba6f241bf8e65325/users/a4fd86cd67d549e888a01c1e735bc1e5/roles/4eea39e24b06431aa357053fe7907a3d => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:25:27.041501 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeb208f-298e-494f-b694-c47ad07d21b6 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:27.042085 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeb208f-298e-494f-b694-c47ad07d21b6 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:27.042547 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeb208f-298e-494f-b694-c47ad07d21b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:27.090071 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-fbeb208f-298e-494f-b694-c47ad07d21b6 None None] MFA Rules not processed for user `a4fd86cd67d549e888a01c1e735bc1e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:25:27.091333 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-fbeb208f-298e-494f-b694-c47ad07d21b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:25:27.183878 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1429/2868] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:25:27 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:25:27.189064 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c7d68a-02f4-4e7a-a1d7-eeb0ae9bdb6e admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:27.296665 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c7d68a-02f4-4e7a-a1d7-eeb0ae9bdb6e admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:27.301901 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8948908e-5980-489a-855e-2a744771ff4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:27.302537 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8948908e-5980-489a-855e-2a744771ff4e admin admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:27.302822 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8948908e-5980-489a-855e-2a744771ff4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:27.303101 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8948908e-5980-489a-855e-2a744771ff4e admin admin] PATH_INFO: `/v3/projects` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:27.303757 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8948908e-5980-489a-855e-2a744771ff4e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:27.305768 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8948908e-5980-489a-855e-2a744771ff4e admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:27.343825 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1440/2869] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:25:27 2022] POST /identity/v3/projects => generated 381 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:25:27.348985 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbeb208f-298e-494f-b694-c47ad07d21b6 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:27.361353 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbeb208f-298e-494f-b694-c47ad07d21b6 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:27.440427 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51be08c0-22ab-43ae-a7ae-fb062617dd54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:27.441489 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51be08c0-22ab-43ae-a7ae-fb062617dd54 admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:27.442130 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51be08c0-22ab-43ae-a7ae-fb062617dd54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:27.442523 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51be08c0-22ab-43ae-a7ae-fb062617dd54 admin admin] PATH_INFO: `/v3/users` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:27.443835 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51be08c0-22ab-43ae-a7ae-fb062617dd54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:27.448270 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51be08c0-22ab-43ae-a7ae-fb062617dd54 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:27.462666 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:25:27.484711 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1430/2870] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:25:27 2022] POST /identity/v3/users => generated 297 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:25:27.489272 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8948908e-5980-489a-855e-2a744771ff4e admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:27.502573 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8948908e-5980-489a-855e-2a744771ff4e admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:27.507405 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4b680a-81cb-4110-a061-5c6a07658793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:27.508040 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4b680a-81cb-4110-a061-5c6a07658793 admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:27.508323 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4b680a-81cb-4110-a061-5c6a07658793 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:27.508602 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4b680a-81cb-4110-a061-5c6a07658793 admin admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:27.509266 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4b680a-81cb-4110-a061-5c6a07658793 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:27.510970 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4b680a-81cb-4110-a061-5c6a07658793 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:27.523557 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1441/2871] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:25:27 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:25:27.528871 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51be08c0-22ab-43ae-a7ae-fb062617dd54 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:27.541061 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51be08c0-22ab-43ae-a7ae-fb062617dd54 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:27.555371 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56e0feb-af56-49c6-9fb5-ab74e22ca908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:27.556222 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56e0feb-af56-49c6-9fb5-ab74e22ca908 admin admin] REQUEST_METHOD: `PUT` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:27.556660 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56e0feb-af56-49c6-9fb5-ab74e22ca908 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:27.557015 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56e0feb-af56-49c6-9fb5-ab74e22ca908 admin admin] PATH_INFO: `/v3/projects/78d4794ab1584d56b383df9b442fe845/users/49045bc7f025450fb45e32115504067f/roles/e0ab41d8ae22474688b5361bcc75e0f0` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:27.581228 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56e0feb-af56-49c6-9fb5-ab74e22ca908 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78d4794ab1584d56b383df9b442fe845, user_id=49045bc7f025450fb45e32115504067f, role_id=e0ab41d8ae22474688b5361bcc75e0f0)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:27.584056 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56e0feb-af56-49c6-9fb5-ab74e22ca908 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:27.602714 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1431/2872] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:25:27 2022] PUT /identity/v3/projects/78d4794ab1584d56b383df9b442fe845/users/49045bc7f025450fb45e32115504067f/roles/e0ab41d8ae22474688b5361bcc75e0f0 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:25:27.605641 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4b680a-81cb-4110-a061-5c6a07658793 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:27.619573 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4b680a-81cb-4110-a061-5c6a07658793 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:27.684232 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a0fd20-72ad-4b94-9644-2b77a7d33ece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:27.684864 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a0fd20-72ad-4b94-9644-2b77a7d33ece admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:27.685362 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a0fd20-72ad-4b94-9644-2b77a7d33ece admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:27.685595 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a0fd20-72ad-4b94-9644-2b77a7d33ece admin admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:27.686409 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a0fd20-72ad-4b94-9644-2b77a7d33ece admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:27.688052 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a0fd20-72ad-4b94-9644-2b77a7d33ece admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:27.700370 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1442/2873] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:25:27 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:25:27.704924 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56e0feb-af56-49c6-9fb5-ab74e22ca908 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:27.716474 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56e0feb-af56-49c6-9fb5-ab74e22ca908 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:27.721824 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6bc65c-1765-44ba-9832-e62cf9e67080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:27.722529 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6bc65c-1765-44ba-9832-e62cf9e67080 admin admin] REQUEST_METHOD: `PUT` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:27.722803 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6bc65c-1765-44ba-9832-e62cf9e67080 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:27.723076 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6bc65c-1765-44ba-9832-e62cf9e67080 admin admin] PATH_INFO: `/v3/projects/78d4794ab1584d56b383df9b442fe845/users/49045bc7f025450fb45e32115504067f/roles/e0ab41d8ae22474688b5361bcc75e0f0` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:27.725156 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6bc65c-1765-44ba-9832-e62cf9e67080 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78d4794ab1584d56b383df9b442fe845, user_id=49045bc7f025450fb45e32115504067f, role_id=e0ab41d8ae22474688b5361bcc75e0f0)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:27.727889 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6bc65c-1765-44ba-9832-e62cf9e67080 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:27.740763 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1432/2874] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:25:27 2022] PUT /identity/v3/projects/78d4794ab1584d56b383df9b442fe845/users/49045bc7f025450fb45e32115504067f/roles/e0ab41d8ae22474688b5361bcc75e0f0 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:25:27.752209 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256b73e8-1cbc-456a-b5d6-bbec51da3237 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:27.752487 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256b73e8-1cbc-456a-b5d6-bbec51da3237 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:27.752818 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256b73e8-1cbc-456a-b5d6-bbec51da3237 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:27.786260 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-256b73e8-1cbc-456a-b5d6-bbec51da3237 None None] MFA Rules not processed for user `49045bc7f025450fb45e32115504067f`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:25:27.787529 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-256b73e8-1cbc-456a-b5d6-bbec51da3237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:25:27.823299 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:25:27.884589 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1443/2875] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:25:27 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:25:27.896428 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6bc65c-1765-44ba-9832-e62cf9e67080 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:27.898214 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-2e6bc65c-1765-44ba-9832-e62cf9e67080 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:25:27.972459 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6bc65c-1765-44ba-9832-e62cf9e67080 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:27.976265 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087a2ed4-b035-4cbd-b628-0e68b1aec44b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df1bca993bc142908cc7cf85eabd2b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:27.976892 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087a2ed4-b035-4cbd-b628-0e68b1aec44b service cinder] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:27.977171 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087a2ed4-b035-4cbd-b628-0e68b1aec44b service cinder] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:27.977444 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087a2ed4-b035-4cbd-b628-0e68b1aec44b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:27.987575 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087a2ed4-b035-4cbd-b628-0e68b1aec44b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:27.989045 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087a2ed4-b035-4cbd-b628-0e68b1aec44b service cinder] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:28.002191 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1433/2876] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:25:27 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 20:25:28.452377 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256b73e8-1cbc-456a-b5d6-bbec51da3237 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:28.530642 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256b73e8-1cbc-456a-b5d6-bbec51da3237 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:28.536507 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11adfcdf-bd01-4173-895d-cee3e8459837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:28.537084 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11adfcdf-bd01-4173-895d-cee3e8459837 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:28.537359 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11adfcdf-bd01-4173-895d-cee3e8459837 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:28.537663 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11adfcdf-bd01-4173-895d-cee3e8459837 admin admin] PATH_INFO: `/v3/users/a4fd86cd67d549e888a01c1e735bc1e5` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:28.538941 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11adfcdf-bd01-4173-895d-cee3e8459837 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4fd86cd67d549e888a01c1e735bc1e5)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:28.541112 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11adfcdf-bd01-4173-895d-cee3e8459837 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:28.606715 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-11adfcdf-bd01-4173-895d-cee3e8459837 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4fd86cd67d549e888a01c1e735bc1e5'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:28.607505 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-11adfcdf-bd01-4173-895d-cee3e8459837 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4fd86cd67d549e888a01c1e735bc1e5'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:28.629036 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-11adfcdf-bd01-4173-895d-cee3e8459837 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4fd86cd67d549e888a01c1e735bc1e5'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:28.630033 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-11adfcdf-bd01-4173-895d-cee3e8459837 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4fd86cd67d549e888a01c1e735bc1e5'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:28.642247 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-11adfcdf-bd01-4173-895d-cee3e8459837 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4fd86cd67d549e888a01c1e735bc1e5'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:28.667551 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1444/2877] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:25:28 2022] DELETE /identity/v3/users/a4fd86cd67d549e888a01c1e735bc1e5 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:25:28.675057 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72397615-2068-4cdc-a733-52cd2497963b None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:28.675689 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72397615-2068-4cdc-a733-52cd2497963b None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:28.676060 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72397615-2068-4cdc-a733-52cd2497963b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:28.707833 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-72397615-2068-4cdc-a733-52cd2497963b None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:25:28.710012 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-72397615-2068-4cdc-a733-52cd2497963b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:25:28.791013 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1434/2878] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:25:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:25:28.802329 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11adfcdf-bd01-4173-895d-cee3e8459837 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:28.803923 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-11adfcdf-bd01-4173-895d-cee3e8459837 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:25:28.870376 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:25:28.879943 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11adfcdf-bd01-4173-895d-cee3e8459837 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:28.884488 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c6dff4-adb0-4e5a-8d99-243a245270c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:28.885238 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c6dff4-adb0-4e5a-8d99-243a245270c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:28.885523 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c6dff4-adb0-4e5a-8d99-243a245270c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:28.885963 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c6dff4-adb0-4e5a-8d99-243a245270c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:28.898168 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c6dff4-adb0-4e5a-8d99-243a245270c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:28.899962 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c6dff4-adb0-4e5a-8d99-243a245270c1 service neutron] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:28.916160 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1445/2879] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:25:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:25:29.275198 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72397615-2068-4cdc-a733-52cd2497963b None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:29.278071 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-72397615-2068-4cdc-a733-52cd2497963b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:25:29.299170 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72397615-2068-4cdc-a733-52cd2497963b None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:29.307526 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5cf5c2-0a2b-4527-9cef-d4804c883193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:29.308583 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5cf5c2-0a2b-4527-9cef-d4804c883193 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:29.309065 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5cf5c2-0a2b-4527-9cef-d4804c883193 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:29.309536 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5cf5c2-0a2b-4527-9cef-d4804c883193 admin admin] PATH_INFO: `/v3/projects/faf1ef13b2424728ba6f241bf8e65325` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:29.311715 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5cf5c2-0a2b-4527-9cef-d4804c883193 admin admin] RBAC: Authorizing `identity:delete_project(project_id=faf1ef13b2424728ba6f241bf8e65325)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:29.316052 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5cf5c2-0a2b-4527-9cef-d4804c883193 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:29.383285 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-fd5cf5c2-0a2b-4527-9cef-d4804c883193 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'faf1ef13b2424728ba6f241bf8e65325', 'request_id': 'req-fd5cf5c2-0a2b-4527-9cef-d4804c883193'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:29.384726 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:INSERT=5,keystone:SELECT=91,keystone:DELETE=4 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:25:29.393321 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-fd5cf5c2-0a2b-4527-9cef-d4804c883193 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'faf1ef13b2424728ba6f241bf8e65325', 'request_id': 'req-fd5cf5c2-0a2b-4527-9cef-d4804c883193'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:29.404732 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-fd5cf5c2-0a2b-4527-9cef-d4804c883193 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'faf1ef13b2424728ba6f241bf8e65325', 'request_id': 'req-fd5cf5c2-0a2b-4527-9cef-d4804c883193'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:29.417705 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-fd5cf5c2-0a2b-4527-9cef-d4804c883193 admin admin] The token cache is being invalidate because project faf1ef13b2424728ba6f241bf8e65325 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81771) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:25:29.418064 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-fd5cf5c2-0a2b-4527-9cef-d4804c883193 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:29.418881 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-fd5cf5c2-0a2b-4527-9cef-d4804c883193 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:29.420639 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1435/2880] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:25:29 2022] DELETE /identity/v3/projects/faf1ef13b2424728ba6f241bf8e65325 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:25:29.426984 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c6dff4-adb0-4e5a-8d99-243a245270c1 service neutron] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:29.439476 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c6dff4-adb0-4e5a-8d99-243a245270c1 service neutron] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:29.516951 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3336f722-4efc-4e96-bd72-12bdf8edee99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:29.518096 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3336f722-4efc-4e96-bd72-12bdf8edee99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:29.518592 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3336f722-4efc-4e96-bd72-12bdf8edee99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:29.519086 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3336f722-4efc-4e96-bd72-12bdf8edee99 admin admin] PATH_INFO: `/v3/users/49045bc7f025450fb45e32115504067f` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:29.521377 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3336f722-4efc-4e96-bd72-12bdf8edee99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=49045bc7f025450fb45e32115504067f)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:29.525252 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3336f722-4efc-4e96-bd72-12bdf8edee99 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:29.599900 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-3336f722-4efc-4e96-bd72-12bdf8edee99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49045bc7f025450fb45e32115504067f'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:29.601096 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-3336f722-4efc-4e96-bd72-12bdf8edee99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49045bc7f025450fb45e32115504067f'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:29.620992 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-3336f722-4efc-4e96-bd72-12bdf8edee99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49045bc7f025450fb45e32115504067f'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:29.622007 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-3336f722-4efc-4e96-bd72-12bdf8edee99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49045bc7f025450fb45e32115504067f'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:29.631678 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-3336f722-4efc-4e96-bd72-12bdf8edee99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49045bc7f025450fb45e32115504067f'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:29.655559 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1446/2881] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:25:29 2022] DELETE /identity/v3/users/49045bc7f025450fb45e32115504067f => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:25:30.099323 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5cf5c2-0a2b-4527-9cef-d4804c883193 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:30.102047 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-fd5cf5c2-0a2b-4527-9cef-d4804c883193 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:25:30.125812 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5cf5c2-0a2b-4527-9cef-d4804c883193 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:30.207877 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847fd014-10be-4287-bdf6-d06c4a98b958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:30.208569 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847fd014-10be-4287-bdf6-d06c4a98b958 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:30.208852 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847fd014-10be-4287-bdf6-d06c4a98b958 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:30.209131 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847fd014-10be-4287-bdf6-d06c4a98b958 admin admin] PATH_INFO: `/v3/projects/78d4794ab1584d56b383df9b442fe845` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:30.210402 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847fd014-10be-4287-bdf6-d06c4a98b958 admin admin] RBAC: Authorizing `identity:delete_project(project_id=78d4794ab1584d56b383df9b442fe845)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:30.212467 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847fd014-10be-4287-bdf6-d06c4a98b958 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:30.275491 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-847fd014-10be-4287-bdf6-d06c4a98b958 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78d4794ab1584d56b383df9b442fe845', 'request_id': 'req-847fd014-10be-4287-bdf6-d06c4a98b958'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:30.284799 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-847fd014-10be-4287-bdf6-d06c4a98b958 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78d4794ab1584d56b383df9b442fe845', 'request_id': 'req-847fd014-10be-4287-bdf6-d06c4a98b958'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:30.294923 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-847fd014-10be-4287-bdf6-d06c4a98b958 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78d4794ab1584d56b383df9b442fe845', 'request_id': 'req-847fd014-10be-4287-bdf6-d06c4a98b958'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:30.309294 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-847fd014-10be-4287-bdf6-d06c4a98b958 admin admin] The token cache is being invalidate because project 78d4794ab1584d56b383df9b442fe845 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81771) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:25:30.309655 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-847fd014-10be-4287-bdf6-d06c4a98b958 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:30.310315 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-847fd014-10be-4287-bdf6-d06c4a98b958 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:30.311971 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1436/2882] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:25:30 2022] DELETE /identity/v3/projects/78d4794ab1584d56b383df9b442fe845 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:25:30.317554 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4a34fc-5240-4895-aa81-e088a9715354 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:30.318032 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4a34fc-5240-4895-aa81-e088a9715354 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:30.318332 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4a34fc-5240-4895-aa81-e088a9715354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:30.342864 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-4d4a34fc-5240-4895-aa81-e088a9715354 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:25:30.344138 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-4d4a34fc-5240-4895-aa81-e088a9715354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:25:30.442690 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1447/2883] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:25:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:25:30.452497 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa47a461-9c8b-45bf-8bb5-0f7e9c120ca7 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:30.452863 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa47a461-9c8b-45bf-8bb5-0f7e9c120ca7 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:30.453178 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa47a461-9c8b-45bf-8bb5-0f7e9c120ca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:30.477638 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-aa47a461-9c8b-45bf-8bb5-0f7e9c120ca7 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:25:30.478925 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-aa47a461-9c8b-45bf-8bb5-0f7e9c120ca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:25:30.489243 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1437/2884] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:25:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 20:25:30.494658 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4a34fc-5240-4895-aa81-e088a9715354 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:30.506952 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4a34fc-5240-4895-aa81-e088a9715354 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:30.512610 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbe3971-467e-466c-8de0-8ca236c9bae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:30.513304 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbe3971-467e-466c-8de0-8ca236c9bae1 admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:30.513656 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbe3971-467e-466c-8de0-8ca236c9bae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:30.513986 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbe3971-467e-466c-8de0-8ca236c9bae1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:30.514791 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbe3971-467e-466c-8de0-8ca236c9bae1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:25:30.515132 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbe3971-467e-466c-8de0-8ca236c9bae1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:30.517325 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbe3971-467e-466c-8de0-8ca236c9bae1 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:30.520624 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:25:30.536081 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1448/2885] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 6 20:25:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:25:30.540190 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa47a461-9c8b-45bf-8bb5-0f7e9c120ca7 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:30.551166 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa47a461-9c8b-45bf-8bb5-0f7e9c120ca7 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:30.557525 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b288d623-3d14-4e89-b592-6e827631f7c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:30.558257 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b288d623-3d14-4e89-b592-6e827631f7c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:30.558540 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b288d623-3d14-4e89-b592-6e827631f7c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:30.558868 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b288d623-3d14-4e89-b592-6e827631f7c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:30.559514 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b288d623-3d14-4e89-b592-6e827631f7c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:30.561879 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b288d623-3d14-4e89-b592-6e827631f7c1 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:30.600153 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1438/2886] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:25:30 2022] POST /identity/v3/projects => generated 385 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 20:25:30.604115 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbe3971-467e-466c-8de0-8ca236c9bae1 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:30.628011 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbe3971-467e-466c-8de0-8ca236c9bae1 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:30.711317 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be54ce9-725b-45d8-9839-de4e3c398dc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:30.712009 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be54ce9-725b-45d8-9839-de4e3c398dc7 admin admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:30.712555 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be54ce9-725b-45d8-9839-de4e3c398dc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:30.712736 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be54ce9-725b-45d8-9839-de4e3c398dc7 admin admin] PATH_INFO: `/v3/users` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:30.713517 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be54ce9-725b-45d8-9839-de4e3c398dc7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:30.715994 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be54ce9-725b-45d8-9839-de4e3c398dc7 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:30.742509 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1449/2887] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:25:30 2022] POST /identity/v3/users => generated 293 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:25:30.747443 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b288d623-3d14-4e89-b592-6e827631f7c1 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:30.760557 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b288d623-3d14-4e89-b592-6e827631f7c1 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:30.766845 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2567262-6053-47af-8e82-c0cfc233ba43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:30.767576 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2567262-6053-47af-8e82-c0cfc233ba43 admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:30.767869 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2567262-6053-47af-8e82-c0cfc233ba43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:30.768163 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2567262-6053-47af-8e82-c0cfc233ba43 admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:30.768846 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2567262-6053-47af-8e82-c0cfc233ba43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:30.770976 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2567262-6053-47af-8e82-c0cfc233ba43 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:30.784256 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1439/2888] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:25:30 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 20:25:30.790402 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be54ce9-725b-45d8-9839-de4e3c398dc7 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:30.802566 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be54ce9-725b-45d8-9839-de4e3c398dc7 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:30.809071 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98199075-6541-425b-b0f2-70282be6337a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:30.810165 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98199075-6541-425b-b0f2-70282be6337a admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:30.810645 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98199075-6541-425b-b0f2-70282be6337a admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:30.811111 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98199075-6541-425b-b0f2-70282be6337a admin admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:30.812454 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98199075-6541-425b-b0f2-70282be6337a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:30.816498 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98199075-6541-425b-b0f2-70282be6337a admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:30.837952 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1450/2889] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:25:30 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:25:30.843344 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2567262-6053-47af-8e82-c0cfc233ba43 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:30.860217 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2567262-6053-47af-8e82-c0cfc233ba43 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:30.866664 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6feb6c08-570f-4d3e-8e22-350f208df6e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:30.867353 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feb6c08-570f-4d3e-8e22-350f208df6e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:30.867811 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feb6c08-570f-4d3e-8e22-350f208df6e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:30.867999 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feb6c08-570f-4d3e-8e22-350f208df6e2 admin admin] PATH_INFO: `/v3/projects/641b5f172dec4e7fa70244f42b3f8102/users/6e7bc0d00ba14a4ca0a620492dc801f6/roles/4eea39e24b06431aa357053fe7907a3d` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:30.887650 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6feb6c08-570f-4d3e-8e22-350f208df6e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=641b5f172dec4e7fa70244f42b3f8102, user_id=6e7bc0d00ba14a4ca0a620492dc801f6, role_id=4eea39e24b06431aa357053fe7907a3d)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:30.890683 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6feb6c08-570f-4d3e-8e22-350f208df6e2 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:30.908400 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1440/2890] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:25:30 2022] PUT /identity/v3/projects/641b5f172dec4e7fa70244f42b3f8102/users/6e7bc0d00ba14a4ca0a620492dc801f6/roles/4eea39e24b06431aa357053fe7907a3d => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:25:30.926708 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8cf50c-a390-4995-9dd0-1b7b313e08cd None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:30.927053 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8cf50c-a390-4995-9dd0-1b7b313e08cd None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:30.927403 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8cf50c-a390-4995-9dd0-1b7b313e08cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:30.962706 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-4d8cf50c-a390-4995-9dd0-1b7b313e08cd None None] MFA Rules not processed for user `6e7bc0d00ba14a4ca0a620492dc801f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:25:30.964051 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-4d8cf50c-a390-4995-9dd0-1b7b313e08cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:25:31.090237 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1451/2891] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:25:30 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:25:31.103146 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6feb6c08-570f-4d3e-8e22-350f208df6e2 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:31.106136 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-6feb6c08-570f-4d3e-8e22-350f208df6e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:25:31.115006 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:25:31.213363 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6feb6c08-570f-4d3e-8e22-350f208df6e2 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:31.219921 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8a7929-d145-46bf-8dab-32bf41278b63 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df1bca993bc142908cc7cf85eabd2b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:31.220968 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8a7929-d145-46bf-8dab-32bf41278b63 service cinder] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:31.221440 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8a7929-d145-46bf-8dab-32bf41278b63 service cinder] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:31.221966 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8a7929-d145-46bf-8dab-32bf41278b63 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:31.239418 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8a7929-d145-46bf-8dab-32bf41278b63 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:31.242254 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8a7929-d145-46bf-8dab-32bf41278b63 service cinder] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:31.260891 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1441/2892] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:25:31 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 20:25:39.199808 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8cf50c-a390-4995-9dd0-1b7b313e08cd None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:39.286159 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8cf50c-a390-4995-9dd0-1b7b313e08cd None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:39.291472 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64e93f8-892e-4fb1-8860-c8cd4b98e9fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:39.292134 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64e93f8-892e-4fb1-8860-c8cd4b98e9fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:39.292428 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64e93f8-892e-4fb1-8860-c8cd4b98e9fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:39.292687 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64e93f8-892e-4fb1-8860-c8cd4b98e9fa admin admin] PATH_INFO: `/v3/users/6e7bc0d00ba14a4ca0a620492dc801f6` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:39.294475 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64e93f8-892e-4fb1-8860-c8cd4b98e9fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e7bc0d00ba14a4ca0a620492dc801f6)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:39.296465 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64e93f8-892e-4fb1-8860-c8cd4b98e9fa admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:39.303203 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:25:39.358296 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-d64e93f8-892e-4fb1-8860-c8cd4b98e9fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e7bc0d00ba14a4ca0a620492dc801f6'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:39.359127 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-d64e93f8-892e-4fb1-8860-c8cd4b98e9fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e7bc0d00ba14a4ca0a620492dc801f6'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:39.374545 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-d64e93f8-892e-4fb1-8860-c8cd4b98e9fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e7bc0d00ba14a4ca0a620492dc801f6'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:39.375331 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-d64e93f8-892e-4fb1-8860-c8cd4b98e9fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e7bc0d00ba14a4ca0a620492dc801f6'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:39.386539 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-d64e93f8-892e-4fb1-8860-c8cd4b98e9fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e7bc0d00ba14a4ca0a620492dc801f6'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:39.402650 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1452/2893] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:25:39 2022] DELETE /identity/v3/users/6e7bc0d00ba14a4ca0a620492dc801f6 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:25:39.407640 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06748bea-953e-4070-b581-4627a7691696 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:39.408560 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06748bea-953e-4070-b581-4627a7691696 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:39.408560 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06748bea-953e-4070-b581-4627a7691696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:39.431914 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-06748bea-953e-4070-b581-4627a7691696 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:25:39.433308 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-06748bea-953e-4070-b581-4627a7691696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:25:39.530830 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1442/2894] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:25:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:25:39.542442 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64e93f8-892e-4fb1-8860-c8cd4b98e9fa admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:39.544111 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-d64e93f8-892e-4fb1-8860-c8cd4b98e9fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:25:39.633119 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64e93f8-892e-4fb1-8860-c8cd4b98e9fa admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:39.640131 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9d64ec-5076-4d7d-bffa-4379bbc77217 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:39.641313 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9d64ec-5076-4d7d-bffa-4379bbc77217 service neutron] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:39.642411 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9d64ec-5076-4d7d-bffa-4379bbc77217 service neutron] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:39.643008 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9d64ec-5076-4d7d-bffa-4379bbc77217 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:39.664745 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9d64ec-5076-4d7d-bffa-4379bbc77217 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:39.667983 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9d64ec-5076-4d7d-bffa-4379bbc77217 service neutron] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:39.695608 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1453/2895] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:25:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 20:25:40.059420 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06748bea-953e-4070-b581-4627a7691696 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:40.061651 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-06748bea-953e-4070-b581-4627a7691696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:25:40.083691 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06748bea-953e-4070-b581-4627a7691696 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:40.092891 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1677ee9b-2858-47ad-bcdd-a36bbe49254c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:40.094458 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1677ee9b-2858-47ad-bcdd-a36bbe49254c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:40.095139 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1677ee9b-2858-47ad-bcdd-a36bbe49254c admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:40.095621 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1677ee9b-2858-47ad-bcdd-a36bbe49254c admin admin] PATH_INFO: `/v3/projects/641b5f172dec4e7fa70244f42b3f8102` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:40.097863 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1677ee9b-2858-47ad-bcdd-a36bbe49254c admin admin] RBAC: Authorizing `identity:delete_project(project_id=641b5f172dec4e7fa70244f42b3f8102)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:40.101321 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1677ee9b-2858-47ad-bcdd-a36bbe49254c admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:40.171204 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-1677ee9b-2858-47ad-bcdd-a36bbe49254c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '641b5f172dec4e7fa70244f42b3f8102', 'request_id': 'req-1677ee9b-2858-47ad-bcdd-a36bbe49254c'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:40.179802 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-1677ee9b-2858-47ad-bcdd-a36bbe49254c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '641b5f172dec4e7fa70244f42b3f8102', 'request_id': 'req-1677ee9b-2858-47ad-bcdd-a36bbe49254c'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:40.190225 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-1677ee9b-2858-47ad-bcdd-a36bbe49254c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '641b5f172dec4e7fa70244f42b3f8102', 'request_id': 'req-1677ee9b-2858-47ad-bcdd-a36bbe49254c'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:40.205177 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-1677ee9b-2858-47ad-bcdd-a36bbe49254c admin admin] The token cache is being invalidate because project 641b5f172dec4e7fa70244f42b3f8102 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81771) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:25:40.205458 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-1677ee9b-2858-47ad-bcdd-a36bbe49254c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:40.206180 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-1677ee9b-2858-47ad-bcdd-a36bbe49254c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:40.208022 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1443/2896] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:25:40 2022] DELETE /identity/v3/projects/641b5f172dec4e7fa70244f42b3f8102 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:25:40.213812 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e23ecec-74db-4710-96b9-070f7228a153 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:40.214654 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e23ecec-74db-4710-96b9-070f7228a153 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:40.214654 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e23ecec-74db-4710-96b9-070f7228a153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:40.238492 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-3e23ecec-74db-4710-96b9-070f7228a153 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:25:40.239757 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-3e23ecec-74db-4710-96b9-070f7228a153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:25:40.324924 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1454/2897] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:25:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:25:40.339271 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8b1942-186c-4648-ba6f-48e67a6e42d5 None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:40.339783 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8b1942-186c-4648-ba6f-48e67a6e42d5 None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:40.340347 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8b1942-186c-4648-ba6f-48e67a6e42d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:40.376007 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-bf8b1942-186c-4648-ba6f-48e67a6e42d5 None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:25:40.377260 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-bf8b1942-186c-4648-ba6f-48e67a6e42d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:25:40.386750 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1444/2898] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:25:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:25:40.392366 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e23ecec-74db-4710-96b9-070f7228a153 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:40.403771 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e23ecec-74db-4710-96b9-070f7228a153 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:40.409097 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731b9b0a-061a-4b38-8787-2fe13928fdc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:40.409776 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731b9b0a-061a-4b38-8787-2fe13928fdc0 admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:40.410057 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731b9b0a-061a-4b38-8787-2fe13928fdc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:40.410326 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731b9b0a-061a-4b38-8787-2fe13928fdc0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:40.411131 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731b9b0a-061a-4b38-8787-2fe13928fdc0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81772) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 20:25:40.411292 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731b9b0a-061a-4b38-8787-2fe13928fdc0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:40.412992 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731b9b0a-061a-4b38-8787-2fe13928fdc0 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:40.429084 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1455/2899] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 6 20:25:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 20:25:40.434214 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8b1942-186c-4648-ba6f-48e67a6e42d5 None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:40.445401 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8b1942-186c-4648-ba6f-48e67a6e42d5 None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:40.451255 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5836f5e0-2e13-4d40-95a7-dbf971187985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:40.451939 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5836f5e0-2e13-4d40-95a7-dbf971187985 admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:40.452265 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5836f5e0-2e13-4d40-95a7-dbf971187985 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:40.452595 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5836f5e0-2e13-4d40-95a7-dbf971187985 admin admin] PATH_INFO: `/v3/projects` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:40.453460 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5836f5e0-2e13-4d40-95a7-dbf971187985 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:40.455734 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5836f5e0-2e13-4d40-95a7-dbf971187985 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:40.491074 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1445/2900] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:25:40 2022] POST /identity/v3/projects => generated 373 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 20:25:40.497829 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731b9b0a-061a-4b38-8787-2fe13928fdc0 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:40.516739 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731b9b0a-061a-4b38-8787-2fe13928fdc0 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:40.534141 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:25:40.628312 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a511d6-fd7a-4b29-81b6-b2041a1a7d6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:40.629892 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a511d6-fd7a-4b29-81b6-b2041a1a7d6d admin admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:40.630407 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a511d6-fd7a-4b29-81b6-b2041a1a7d6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:40.630907 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a511d6-fd7a-4b29-81b6-b2041a1a7d6d admin admin] PATH_INFO: `/v3/users` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:40.632111 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a511d6-fd7a-4b29-81b6-b2041a1a7d6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:40.636186 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a511d6-fd7a-4b29-81b6-b2041a1a7d6d admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:40.672875 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1456/2901] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:25:40 2022] POST /identity/v3/users => generated 287 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:25:40.677412 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5836f5e0-2e13-4d40-95a7-dbf971187985 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:40.683866 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:25:40.691286 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5836f5e0-2e13-4d40-95a7-dbf971187985 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:40.698317 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9a560a-88e5-4f75-9882-5d3d0c8eb331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:40.699092 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9a560a-88e5-4f75-9882-5d3d0c8eb331 admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:40.699723 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9a560a-88e5-4f75-9882-5d3d0c8eb331 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:40.700276 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9a560a-88e5-4f75-9882-5d3d0c8eb331 admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:40.701452 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9a560a-88e5-4f75-9882-5d3d0c8eb331 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:40.703542 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9a560a-88e5-4f75-9882-5d3d0c8eb331 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:40.715982 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1446/2902] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:25:40 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 20:25:40.720602 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a511d6-fd7a-4b29-81b6-b2041a1a7d6d admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:40.731744 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a511d6-fd7a-4b29-81b6-b2041a1a7d6d admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:40.738246 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d57941d-d879-4ce6-906d-58931ec86b55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:40.738766 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d57941d-d879-4ce6-906d-58931ec86b55 admin admin] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:40.739048 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d57941d-d879-4ce6-906d-58931ec86b55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:40.739321 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d57941d-d879-4ce6-906d-58931ec86b55 admin admin] PATH_INFO: `/v3/roles` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:40.739988 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d57941d-d879-4ce6-906d-58931ec86b55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:40.741673 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d57941d-d879-4ce6-906d-58931ec86b55 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:40.754953 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1457/2903] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:25:40 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:25:40.762323 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9a560a-88e5-4f75-9882-5d3d0c8eb331 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:40.781998 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9a560a-88e5-4f75-9882-5d3d0c8eb331 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:40.795426 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09520be4-293b-47fc-8ab2-4a4c920cd6f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:40.796523 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09520be4-293b-47fc-8ab2-4a4c920cd6f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:40.797083 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09520be4-293b-47fc-8ab2-4a4c920cd6f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:40.797572 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09520be4-293b-47fc-8ab2-4a4c920cd6f0 admin admin] PATH_INFO: `/v3/projects/6795c5f0b32f42349402e096621aa790/users/470dbd3a3d05444e9d98f7e1ebc9c61b/roles/4eea39e24b06431aa357053fe7907a3d` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:40.818209 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09520be4-293b-47fc-8ab2-4a4c920cd6f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6795c5f0b32f42349402e096621aa790, user_id=470dbd3a3d05444e9d98f7e1ebc9c61b, role_id=4eea39e24b06431aa357053fe7907a3d)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:40.821028 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09520be4-293b-47fc-8ab2-4a4c920cd6f0 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:40.837658 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1447/2904] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:25:40 2022] PUT /identity/v3/projects/6795c5f0b32f42349402e096621aa790/users/470dbd3a3d05444e9d98f7e1ebc9c61b/roles/4eea39e24b06431aa357053fe7907a3d => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:25:40.850701 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ca8d91-d494-4c7c-8711-691780a9b6f1 None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:40.851059 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ca8d91-d494-4c7c-8711-691780a9b6f1 None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:40.851419 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ca8d91-d494-4c7c-8711-691780a9b6f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:40.886934 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-67ca8d91-d494-4c7c-8711-691780a9b6f1 None None] MFA Rules not processed for user `470dbd3a3d05444e9d98f7e1ebc9c61b`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:25:40.888369 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-67ca8d91-d494-4c7c-8711-691780a9b6f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:25:40.994258 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1458/2905] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:25:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:25:41.000009 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09520be4-293b-47fc-8ab2-4a4c920cd6f0 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:41.095345 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09520be4-293b-47fc-8ab2-4a4c920cd6f0 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:41.102252 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98bdb90-3ae5-474c-a39d-a4cb5b4d1c40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:41.102887 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98bdb90-3ae5-474c-a39d-a4cb5b4d1c40 admin admin] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:41.103166 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98bdb90-3ae5-474c-a39d-a4cb5b4d1c40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:41.103438 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98bdb90-3ae5-474c-a39d-a4cb5b4d1c40 admin admin] PATH_INFO: `/v3/projects` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:41.104110 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98bdb90-3ae5-474c-a39d-a4cb5b4d1c40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:41.106234 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98bdb90-3ae5-474c-a39d-a4cb5b4d1c40 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:41.146759 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1448/2906] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 6 20:25:40 2022] POST /identity/v3/projects => generated 373 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 20:25:41.151458 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ca8d91-d494-4c7c-8711-691780a9b6f1 None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:41.168119 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ca8d91-d494-4c7c-8711-691780a9b6f1 None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:41.246180 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14c4503-f183-41ca-9b0e-210246ce5559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:41.246180 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14c4503-f183-41ca-9b0e-210246ce5559 admin admin] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:41.247407 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14c4503-f183-41ca-9b0e-210246ce5559 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:41.247407 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14c4503-f183-41ca-9b0e-210246ce5559 admin admin] PATH_INFO: `/v3/users` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:41.247787 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14c4503-f183-41ca-9b0e-210246ce5559 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:41.251394 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14c4503-f183-41ca-9b0e-210246ce5559 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:41.278116 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1459/2907] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 6 20:25:41 2022] POST /identity/v3/users => generated 293 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 20:25:41.283488 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98bdb90-3ae5-474c-a39d-a4cb5b4d1c40 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:41.295103 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98bdb90-3ae5-474c-a39d-a4cb5b4d1c40 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:41.300432 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b58cb3-0fb5-4214-8b0b-b6c8616170d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:41.301072 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b58cb3-0fb5-4214-8b0b-b6c8616170d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:41.301352 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b58cb3-0fb5-4214-8b0b-b6c8616170d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:41.301644 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b58cb3-0fb5-4214-8b0b-b6c8616170d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:41.302348 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b58cb3-0fb5-4214-8b0b-b6c8616170d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:41.304070 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b58cb3-0fb5-4214-8b0b-b6c8616170d1 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:41.318721 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1449/2908] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:25:41 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 20:25:41.323293 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14c4503-f183-41ca-9b0e-210246ce5559 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:41.334787 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14c4503-f183-41ca-9b0e-210246ce5559 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:41.340327 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9004de-73b7-486d-ba53-7961446c7dfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:41.341168 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9004de-73b7-486d-ba53-7961446c7dfd admin admin] REQUEST_METHOD: `PUT` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:41.341554 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9004de-73b7-486d-ba53-7961446c7dfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:41.342026 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9004de-73b7-486d-ba53-7961446c7dfd admin admin] PATH_INFO: `/v3/projects/9b99ec740c774c7c85f8162667ab06bd/users/e1404f1eae784c5fb236a8f79a73f8a6/roles/e0ab41d8ae22474688b5361bcc75e0f0` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:41.354354 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:25:41.364540 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9004de-73b7-486d-ba53-7961446c7dfd admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b99ec740c774c7c85f8162667ab06bd, user_id=e1404f1eae784c5fb236a8f79a73f8a6, role_id=e0ab41d8ae22474688b5361bcc75e0f0)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:41.367878 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9004de-73b7-486d-ba53-7961446c7dfd admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:41.385794 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1460/2909] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:25:41 2022] PUT /identity/v3/projects/9b99ec740c774c7c85f8162667ab06bd/users/e1404f1eae784c5fb236a8f79a73f8a6/roles/e0ab41d8ae22474688b5361bcc75e0f0 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 20:25:41.390583 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b58cb3-0fb5-4214-8b0b-b6c8616170d1 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:41.401865 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b58cb3-0fb5-4214-8b0b-b6c8616170d1 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:41.479512 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d65f25-f5bb-4a83-864e-53b1ea04a6e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:41.480322 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d65f25-f5bb-4a83-864e-53b1ea04a6e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:41.480794 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d65f25-f5bb-4a83-864e-53b1ea04a6e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:41.481122 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d65f25-f5bb-4a83-864e-53b1ea04a6e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:41.481899 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d65f25-f5bb-4a83-864e-53b1ea04a6e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:41.483642 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d65f25-f5bb-4a83-864e-53b1ea04a6e5 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:41.496158 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1450/2910] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 6 20:25:41 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 20:25:41.501590 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9004de-73b7-486d-ba53-7961446c7dfd admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:41.514065 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9004de-73b7-486d-ba53-7961446c7dfd admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:41.520119 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a53d25-2573-4f38-807e-bcb6ede84abe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:41.520934 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a53d25-2573-4f38-807e-bcb6ede84abe admin admin] REQUEST_METHOD: `PUT` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:41.521224 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a53d25-2573-4f38-807e-bcb6ede84abe admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:41.521502 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a53d25-2573-4f38-807e-bcb6ede84abe admin admin] PATH_INFO: `/v3/projects/9b99ec740c774c7c85f8162667ab06bd/users/e1404f1eae784c5fb236a8f79a73f8a6/roles/e0ab41d8ae22474688b5361bcc75e0f0` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:41.523808 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a53d25-2573-4f38-807e-bcb6ede84abe admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b99ec740c774c7c85f8162667ab06bd, user_id=e1404f1eae784c5fb236a8f79a73f8a6, role_id=e0ab41d8ae22474688b5361bcc75e0f0)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:41.526659 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a53d25-2573-4f38-807e-bcb6ede84abe admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:41.538760 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1461/2911] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 6 20:25:41 2022] PUT /identity/v3/projects/9b99ec740c774c7c85f8162667ab06bd/users/e1404f1eae784c5fb236a8f79a73f8a6/roles/e0ab41d8ae22474688b5361bcc75e0f0 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:25:41.551062 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16b5091-c457-4353-a6eb-5809554d8b4e None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:41.551507 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16b5091-c457-4353-a6eb-5809554d8b4e None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:41.551944 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16b5091-c457-4353-a6eb-5809554d8b4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:41.600009 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-a16b5091-c457-4353-a6eb-5809554d8b4e None None] MFA Rules not processed for user `e1404f1eae784c5fb236a8f79a73f8a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:25:41.601847 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-a16b5091-c457-4353-a6eb-5809554d8b4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:25:41.649021 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:25:41.722914 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1451/2912] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:25:41 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:25:41.734526 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a53d25-2573-4f38-807e-bcb6ede84abe admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:41.737605 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-47a53d25-2573-4f38-807e-bcb6ede84abe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:25:41.830790 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a53d25-2573-4f38-807e-bcb6ede84abe admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:41.835066 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d594e5b7-7fbd-4e6c-b074-4cba7dd95955 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df1bca993bc142908cc7cf85eabd2b24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:41.835833 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d594e5b7-7fbd-4e6c-b074-4cba7dd95955 service cinder] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:41.836157 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d594e5b7-7fbd-4e6c-b074-4cba7dd95955 service cinder] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:41.836579 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d594e5b7-7fbd-4e6c-b074-4cba7dd95955 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:41.914970 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d594e5b7-7fbd-4e6c-b074-4cba7dd95955 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:41.914970 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d594e5b7-7fbd-4e6c-b074-4cba7dd95955 service cinder] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:41.926158 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1462/2913] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:25:41 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:25:44.729867 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16b5091-c457-4353-a6eb-5809554d8b4e None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:44.809311 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16b5091-c457-4353-a6eb-5809554d8b4e None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:44.815089 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6eadb0-c641-46cd-929d-b82ac3190f2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:44.815709 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6eadb0-c641-46cd-929d-b82ac3190f2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:44.816036 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6eadb0-c641-46cd-929d-b82ac3190f2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:44.816339 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6eadb0-c641-46cd-929d-b82ac3190f2d admin admin] PATH_INFO: `/v3/users/470dbd3a3d05444e9d98f7e1ebc9c61b` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:44.817836 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6eadb0-c641-46cd-929d-b82ac3190f2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=470dbd3a3d05444e9d98f7e1ebc9c61b)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:44.819967 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6eadb0-c641-46cd-929d-b82ac3190f2d admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:44.884227 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-8e6eadb0-c641-46cd-929d-b82ac3190f2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '470dbd3a3d05444e9d98f7e1ebc9c61b'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:44.895023 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-8e6eadb0-c641-46cd-929d-b82ac3190f2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '470dbd3a3d05444e9d98f7e1ebc9c61b'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:44.895778 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-8e6eadb0-c641-46cd-929d-b82ac3190f2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '470dbd3a3d05444e9d98f7e1ebc9c61b'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:44.896459 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-8e6eadb0-c641-46cd-929d-b82ac3190f2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '470dbd3a3d05444e9d98f7e1ebc9c61b'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:44.907308 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-8e6eadb0-c641-46cd-929d-b82ac3190f2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '470dbd3a3d05444e9d98f7e1ebc9c61b'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:44.932944 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1452/2914] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:25:44 2022] DELETE /identity/v3/users/470dbd3a3d05444e9d98f7e1ebc9c61b => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:25:44.940262 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63cae13-395b-4a11-b46f-2cfca2b675bf None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:44.940910 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63cae13-395b-4a11-b46f-2cfca2b675bf None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:44.941168 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63cae13-395b-4a11-b46f-2cfca2b675bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:44.974103 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-e63cae13-395b-4a11-b46f-2cfca2b675bf None None] MFA Rules not processed for user `1e7095a0137b45c984179dcf25de9362`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:25:44.976295 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-e63cae13-395b-4a11-b46f-2cfca2b675bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:25:45.093933 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1463/2915] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:25:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:25:45.109878 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6eadb0-c641-46cd-929d-b82ac3190f2d admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:45.111770 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-8e6eadb0-c641-46cd-929d-b82ac3190f2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:25:45.172303 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:25:45.190198 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6eadb0-c641-46cd-929d-b82ac3190f2d admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:45.195056 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c25dd0f-5c07-4b55-a73d-43ec05db8fec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:45.195811 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c25dd0f-5c07-4b55-a73d-43ec05db8fec service neutron] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:45.196188 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c25dd0f-5c07-4b55-a73d-43ec05db8fec service neutron] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:45.196506 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c25dd0f-5c07-4b55-a73d-43ec05db8fec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:45.207565 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c25dd0f-5c07-4b55-a73d-43ec05db8fec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:45.209142 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c25dd0f-5c07-4b55-a73d-43ec05db8fec service neutron] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:45.222288 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1453/2916] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:25:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:25:45.561972 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63cae13-395b-4a11-b46f-2cfca2b675bf None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:45.566054 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-e63cae13-395b-4a11-b46f-2cfca2b675bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:25:45.589023 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63cae13-395b-4a11-b46f-2cfca2b675bf None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:45.598503 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbff28ad-749e-4244-a230-ce4954056bef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:45.599665 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbff28ad-749e-4244-a230-ce4954056bef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:45.600147 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbff28ad-749e-4244-a230-ce4954056bef admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:45.600599 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbff28ad-749e-4244-a230-ce4954056bef admin admin] PATH_INFO: `/v3/projects/6795c5f0b32f42349402e096621aa790` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:45.602769 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbff28ad-749e-4244-a230-ce4954056bef admin admin] RBAC: Authorizing `identity:delete_project(project_id=6795c5f0b32f42349402e096621aa790)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:45.606645 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbff28ad-749e-4244-a230-ce4954056bef admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:45.656535 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:25:45.695772 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-bbff28ad-749e-4244-a230-ce4954056bef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6795c5f0b32f42349402e096621aa790', 'request_id': 'req-bbff28ad-749e-4244-a230-ce4954056bef'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:45.709079 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-bbff28ad-749e-4244-a230-ce4954056bef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6795c5f0b32f42349402e096621aa790', 'request_id': 'req-bbff28ad-749e-4244-a230-ce4954056bef'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:45.717823 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-bbff28ad-749e-4244-a230-ce4954056bef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6795c5f0b32f42349402e096621aa790', 'request_id': 'req-bbff28ad-749e-4244-a230-ce4954056bef'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:45.732401 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-bbff28ad-749e-4244-a230-ce4954056bef admin admin] The token cache is being invalidate because project 6795c5f0b32f42349402e096621aa790 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81772) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:25:45.732761 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-bbff28ad-749e-4244-a230-ce4954056bef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:45.733577 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-bbff28ad-749e-4244-a230-ce4954056bef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:45.735456 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1464/2917] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:25:45 2022] DELETE /identity/v3/projects/6795c5f0b32f42349402e096621aa790 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:25:45.739957 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c25dd0f-5c07-4b55-a73d-43ec05db8fec service neutron] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:45.751949 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c25dd0f-5c07-4b55-a73d-43ec05db8fec service neutron] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:45.823255 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6794e4f7-bd7f-494f-9b3c-fefcb7bd6a02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:45.823986 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6794e4f7-bd7f-494f-9b3c-fefcb7bd6a02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:45.824334 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6794e4f7-bd7f-494f-9b3c-fefcb7bd6a02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:45.824674 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6794e4f7-bd7f-494f-9b3c-fefcb7bd6a02 admin admin] PATH_INFO: `/v3/users/e1404f1eae784c5fb236a8f79a73f8a6` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:45.825895 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6794e4f7-bd7f-494f-9b3c-fefcb7bd6a02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1404f1eae784c5fb236a8f79a73f8a6)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:45.827971 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6794e4f7-bd7f-494f-9b3c-fefcb7bd6a02 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:45.892075 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-6794e4f7-bd7f-494f-9b3c-fefcb7bd6a02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1404f1eae784c5fb236a8f79a73f8a6'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:45.901158 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-6794e4f7-bd7f-494f-9b3c-fefcb7bd6a02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1404f1eae784c5fb236a8f79a73f8a6'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:45.901913 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-6794e4f7-bd7f-494f-9b3c-fefcb7bd6a02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1404f1eae784c5fb236a8f79a73f8a6'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:45.902876 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-6794e4f7-bd7f-494f-9b3c-fefcb7bd6a02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1404f1eae784c5fb236a8f79a73f8a6'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:45.920177 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-6794e4f7-bd7f-494f-9b3c-fefcb7bd6a02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1404f1eae784c5fb236a8f79a73f8a6'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:45.953819 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1454/2918] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:25:45 2022] DELETE /identity/v3/users/e1404f1eae784c5fb236a8f79a73f8a6 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:25:46.370290 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbff28ad-749e-4244-a230-ce4954056bef admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:46.372391 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-bbff28ad-749e-4244-a230-ce4954056bef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:25:46.395568 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbff28ad-749e-4244-a230-ce4954056bef admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:46.479108 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7535506a-17ec-459e-bc59-9142191bb3e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:46.479828 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7535506a-17ec-459e-bc59-9142191bb3e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:46.480232 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7535506a-17ec-459e-bc59-9142191bb3e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:46.480622 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7535506a-17ec-459e-bc59-9142191bb3e9 admin admin] PATH_INFO: `/v3/projects/9b99ec740c774c7c85f8162667ab06bd` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:46.482779 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7535506a-17ec-459e-bc59-9142191bb3e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b99ec740c774c7c85f8162667ab06bd)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:46.485352 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7535506a-17ec-459e-bc59-9142191bb3e9 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:46.553144 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-7535506a-17ec-459e-bc59-9142191bb3e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b99ec740c774c7c85f8162667ab06bd', 'request_id': 'req-7535506a-17ec-459e-bc59-9142191bb3e9'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:46.564738 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-7535506a-17ec-459e-bc59-9142191bb3e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b99ec740c774c7c85f8162667ab06bd', 'request_id': 'req-7535506a-17ec-459e-bc59-9142191bb3e9'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:46.573306 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-7535506a-17ec-459e-bc59-9142191bb3e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b99ec740c774c7c85f8162667ab06bd', 'request_id': 'req-7535506a-17ec-459e-bc59-9142191bb3e9'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:46.585558 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-7535506a-17ec-459e-bc59-9142191bb3e9 admin admin] The token cache is being invalidate because project 9b99ec740c774c7c85f8162667ab06bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81772) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:25:46.586107 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-7535506a-17ec-459e-bc59-9142191bb3e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:46.587374 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-7535506a-17ec-459e-bc59-9142191bb3e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:25:46.589258 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1465/2919] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:25:46 2022] DELETE /identity/v3/projects/9b99ec740c774c7c85f8162667ab06bd => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:25:55.946634 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=60,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:25:56.582110 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=66,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:25:56.684089 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6794e4f7-bd7f-494f-9b3c-fefcb7bd6a02 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:25:56.686081 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-6794e4f7-bd7f-494f-9b3c-fefcb7bd6a02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:25:56.760537 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6794e4f7-bd7f-494f-9b3c-fefcb7bd6a02 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:25:56.824292 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d71c83-9c10-45b9-af68-9b4eeabe0292 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:25:56.825085 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d71c83-9c10-45b9-af68-9b4eeabe0292 service neutron] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:25:56.825471 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d71c83-9c10-45b9-af68-9b4eeabe0292 service neutron] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:25:56.825909 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d71c83-9c10-45b9-af68-9b4eeabe0292 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:25:56.828087 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-d6d71c83-9c10-45b9-af68-9b4eeabe0292 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:25:56.841705 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d71c83-9c10-45b9-af68-9b4eeabe0292 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:25:56.843365 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d71c83-9c10-45b9-af68-9b4eeabe0292 service neutron] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:25:56.854732 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1455/2920] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:25:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:26:00.509519 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7535506a-17ec-459e-bc59-9142191bb3e9 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:26:00.511359 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-7535506a-17ec-459e-bc59-9142191bb3e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:26:00.525354 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7535506a-17ec-459e-bc59-9142191bb3e9 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:26:00.530667 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a06da5-17e8-410d-a32b-b7bc96630a63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:26:00.531303 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a06da5-17e8-410d-a32b-b7bc96630a63 service neutron] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:26:00.531585 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a06da5-17e8-410d-a32b-b7bc96630a63 service neutron] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:26:00.531858 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a06da5-17e8-410d-a32b-b7bc96630a63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:26:00.534082 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-b0a06da5-17e8-410d-a32b-b7bc96630a63 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:26:00.629861 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a06da5-17e8-410d-a32b-b7bc96630a63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:26:00.631408 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a06da5-17e8-410d-a32b-b7bc96630a63 service neutron] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:26:00.650017 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1466/2921] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:26:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:26:02.183410 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d71c83-9c10-45b9-af68-9b4eeabe0292 service neutron] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:26:02.185753 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-d6d71c83-9c10-45b9-af68-9b4eeabe0292 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:26:02.226246 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d71c83-9c10-45b9-af68-9b4eeabe0292 service neutron] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:26:02.238818 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee352f71-9a49-440a-ac43-712bf0dfd378 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5776cbf4f6714b2083c3ebdd6d98cc87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:26:02.240788 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee352f71-9a49-440a-ac43-712bf0dfd378 service nova] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:26:02.241319 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee352f71-9a49-440a-ac43-712bf0dfd378 service nova] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:26:02.242200 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee352f71-9a49-440a-ac43-712bf0dfd378 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:26:02.249136 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-ee352f71-9a49-440a-ac43-712bf0dfd378 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:26:02.278190 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee352f71-9a49-440a-ac43-712bf0dfd378 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:26:02.286669 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee352f71-9a49-440a-ac43-712bf0dfd378 service nova] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:26:02.304845 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1456/2922] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:26:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:26:04.120018 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b13c1b-da4b-43fc-8a25-cd6501358f8b None None] REQUEST_METHOD: `POST` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:26:04.120837 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b13c1b-da4b-43fc-8a25-cd6501358f8b None None] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:26:04.120837 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b13c1b-da4b-43fc-8a25-cd6501358f8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:26:04.144718 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.auth.core [None req-58b13c1b-da4b-43fc-8a25-cd6501358f8b None None] MFA Rules not processed for user `7f661aa1f00e4c7e916b2498505475dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81772) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:26:04.146039 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-58b13c1b-da4b-43fc-8a25-cd6501358f8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:26:04.236898 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1467/2923] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:26:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:26:04.637207 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee352f71-9a49-440a-ac43-712bf0dfd378 service nova] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:26:04.639470 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-ee352f71-9a49-440a-ac43-712bf0dfd378 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:26:04.658341 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee352f71-9a49-440a-ac43-712bf0dfd378 service nova] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:26:04.662817 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eefbe27-3a1d-45a0-9963-767f98f1ddd6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:26:04.663458 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eefbe27-3a1d-45a0-9963-767f98f1ddd6 service neutron] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:26:04.663743 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eefbe27-3a1d-45a0-9963-767f98f1ddd6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:26:04.664023 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eefbe27-3a1d-45a0-9963-767f98f1ddd6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:26:04.666065 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-3eefbe27-3a1d-45a0-9963-767f98f1ddd6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:26:04.678152 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eefbe27-3a1d-45a0-9963-767f98f1ddd6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:26:04.679894 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eefbe27-3a1d-45a0-9963-767f98f1ddd6 service neutron] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:26:04.690874 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1457/2924] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:26:04 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:26:05.298283 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b13c1b-da4b-43fc-8a25-cd6501358f8b None None] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:26:05.299846 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-58b13c1b-da4b-43fc-8a25-cd6501358f8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:26:05.408987 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b13c1b-da4b-43fc-8a25-cd6501358f8b None None] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:26:05.416117 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd72263-67e9-41b3-9e44-f2ce3a1954df service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f57872de7bb74ea48d30e93e51830aba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:26:05.416807 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd72263-67e9-41b3-9e44-f2ce3a1954df service placement] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:26:05.417152 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd72263-67e9-41b3-9e44-f2ce3a1954df service placement] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:26:05.417601 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd72263-67e9-41b3-9e44-f2ce3a1954df service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:26:05.419790 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-4cd72263-67e9-41b3-9e44-f2ce3a1954df service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:26:05.434769 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd72263-67e9-41b3-9e44-f2ce3a1954df service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:26:05.436542 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd72263-67e9-41b3-9e44-f2ce3a1954df service placement] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:26:05.449534 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1468/2925] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:26:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:26:14.685749 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=58 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:26:15.444118 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=91 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:26:20.089043 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eefbe27-3a1d-45a0-9963-767f98f1ddd6 service neutron] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:26:20.092669 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-3eefbe27-3a1d-45a0-9963-767f98f1ddd6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:26:20.131270 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eefbe27-3a1d-45a0-9963-767f98f1ddd6 service neutron] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:26:20.139386 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950481f1-dc4a-49fd-a4bd-79b7d023ee3f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5776cbf4f6714b2083c3ebdd6d98cc87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:26:20.140312 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950481f1-dc4a-49fd-a4bd-79b7d023ee3f service nova] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:26:20.141329 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950481f1-dc4a-49fd-a4bd-79b7d023ee3f service nova] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:26:20.141848 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950481f1-dc4a-49fd-a4bd-79b7d023ee3f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:26:20.144691 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-950481f1-dc4a-49fd-a4bd-79b7d023ee3f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:26:20.162869 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950481f1-dc4a-49fd-a4bd-79b7d023ee3f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:26:20.165355 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950481f1-dc4a-49fd-a4bd-79b7d023ee3f service nova] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:26:20.184019 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1458/2926] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:26:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:26:25.885182 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd72263-67e9-41b3-9e44-f2ce3a1954df service placement] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:26:25.902473 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd72263-67e9-41b3-9e44-f2ce3a1954df service placement] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:26:25.914002 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40590edf-9007-4918-89aa-4c5b6df20898 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5776cbf4f6714b2083c3ebdd6d98cc87', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:26:25.914801 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40590edf-9007-4918-89aa-4c5b6df20898 service nova] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:26:25.915085 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40590edf-9007-4918-89aa-4c5b6df20898 service nova] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:26:25.915361 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40590edf-9007-4918-89aa-4c5b6df20898 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:26:25.917388 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-40590edf-9007-4918-89aa-4c5b6df20898 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:26:25.933054 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40590edf-9007-4918-89aa-4c5b6df20898 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:26:25.934954 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40590edf-9007-4918-89aa-4c5b6df20898 service nova] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:26:25.950001 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1469/2927] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:26:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:26:30.178181 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=6 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:26:35.943898 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=6 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:28:05.142833 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34987ba1-966f-42a7-9925-e5096242eacd None None] REQUEST_METHOD: `POST` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:28:05.143482 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34987ba1-966f-42a7-9925-e5096242eacd None None] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:28:05.143857 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34987ba1-966f-42a7-9925-e5096242eacd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:28:05.177713 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.auth.core [None req-34987ba1-966f-42a7-9925-e5096242eacd None None] MFA Rules not processed for user `7f661aa1f00e4c7e916b2498505475dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81771) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 20:28:05.179903 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-34987ba1-966f-42a7-9925-e5096242eacd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:28:05.193006 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1459/2928] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 6 20:28:05 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 20:28:05.611993 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40590edf-9007-4918-89aa-4c5b6df20898 service nova] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:28:05.623807 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40590edf-9007-4918-89aa-4c5b6df20898 service nova] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:28:05.628126 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af6f944-23ea-4abc-9b80-d510ecffdaea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:28:05.628807 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af6f944-23ea-4abc-9b80-d510ecffdaea service neutron] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:28:05.629132 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af6f944-23ea-4abc-9b80-d510ecffdaea service neutron] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:28:05.629448 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af6f944-23ea-4abc-9b80-d510ecffdaea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:28:05.631486 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-1af6f944-23ea-4abc-9b80-d510ecffdaea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:28:05.646331 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af6f944-23ea-4abc-9b80-d510ecffdaea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:28:05.647936 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af6f944-23ea-4abc-9b80-d510ecffdaea service neutron] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:28:05.658535 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1470/2929] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:28:05 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:28:07.780067 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34987ba1-966f-42a7-9925-e5096242eacd None None] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:28:07.796575 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34987ba1-966f-42a7-9925-e5096242eacd None None] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:28:07.802244 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daac5c43-94ae-4436-847e-17ef7b1093f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:28:07.803270 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daac5c43-94ae-4436-847e-17ef7b1093f0 service neutron] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:28:07.805532 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daac5c43-94ae-4436-847e-17ef7b1093f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:28:07.806054 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daac5c43-94ae-4436-847e-17ef7b1093f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:28:07.808508 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-daac5c43-94ae-4436-847e-17ef7b1093f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:28:07.825419 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daac5c43-94ae-4436-847e-17ef7b1093f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:28:07.828174 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daac5c43-94ae-4436-847e-17ef7b1093f0 service neutron] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:28:07.840419 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1460/2930] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:28:07 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:28:15.653334 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=6 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:28:17.834428 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=9 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:28:58.963820 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af6f944-23ea-4abc-9b80-d510ecffdaea service neutron] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:28:58.965642 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-1af6f944-23ea-4abc-9b80-d510ecffdaea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:28:58.983552 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af6f944-23ea-4abc-9b80-d510ecffdaea service neutron] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:28:58.989696 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c585d5-11b3-45bd-a665-89e1aad0bd99 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f57872de7bb74ea48d30e93e51830aba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:28:58.990328 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c585d5-11b3-45bd-a665-89e1aad0bd99 service placement] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:28:58.990613 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c585d5-11b3-45bd-a665-89e1aad0bd99 service placement] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:28:58.990889 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c585d5-11b3-45bd-a665-89e1aad0bd99 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:28:58.992798 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-c0c585d5-11b3-45bd-a665-89e1aad0bd99 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:28:59.006917 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c585d5-11b3-45bd-a665-89e1aad0bd99 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:28:59.008799 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c585d5-11b3-45bd-a665-89e1aad0bd99 service placement] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:28:59.022028 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1471/2931] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:28:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:29:09.016576 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=6 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:30:14.577871 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daac5c43-94ae-4436-847e-17ef7b1093f0 service neutron] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:30:14.598143 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daac5c43-94ae-4436-847e-17ef7b1093f0 service neutron] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:30:14.605485 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136ce97e-cc51-44af-97a1-65e1fceb9553 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:30:14.607702 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136ce97e-cc51-44af-97a1-65e1fceb9553 service neutron] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:30:14.607702 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136ce97e-cc51-44af-97a1-65e1fceb9553 service neutron] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:30:14.608189 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136ce97e-cc51-44af-97a1-65e1fceb9553 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:30:14.610699 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-136ce97e-cc51-44af-97a1-65e1fceb9553 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:30:14.639306 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136ce97e-cc51-44af-97a1-65e1fceb9553 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:30:14.641760 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136ce97e-cc51-44af-97a1-65e1fceb9553 service neutron] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:30:14.665958 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1461/2932] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:30:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:30:19.613993 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c585d5-11b3-45bd-a665-89e1aad0bd99 service placement] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:30:19.628814 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c585d5-11b3-45bd-a665-89e1aad0bd99 service placement] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:30:19.633120 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb3b9e0-833e-44d1-9001-aa90996ee161 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32882d38295046fea004a659a71bde58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:30:19.633961 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb3b9e0-833e-44d1-9001-aa90996ee161 service neutron] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:30:19.634335 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb3b9e0-833e-44d1-9001-aa90996ee161 service neutron] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:30:19.634677 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb3b9e0-833e-44d1-9001-aa90996ee161 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:30:19.636967 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-3fb3b9e0-833e-44d1-9001-aa90996ee161 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:30:19.724702 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb3b9e0-833e-44d1-9001-aa90996ee161 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:30:19.726238 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb3b9e0-833e-44d1-9001-aa90996ee161 service neutron] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:30:19.738205 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1472/2933] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:30:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:30:24.155597 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136ce97e-cc51-44af-97a1-65e1fceb9553 service neutron] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:30:24.177340 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136ce97e-cc51-44af-97a1-65e1fceb9553 service neutron] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:30:24.188734 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee79bddd-5720-40f0-b651-de1ae9e8979e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f57872de7bb74ea48d30e93e51830aba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:30:24.189544 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee79bddd-5720-40f0-b651-de1ae9e8979e service placement] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:30:24.190024 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee79bddd-5720-40f0-b651-de1ae9e8979e service placement] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:30:24.190433 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee79bddd-5720-40f0-b651-de1ae9e8979e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:30:24.192724 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-ee79bddd-5720-40f0-b651-de1ae9e8979e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:30:24.208074 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee79bddd-5720-40f0-b651-de1ae9e8979e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:30:24.211292 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee79bddd-5720-40f0-b651-de1ae9e8979e service placement] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:30:24.241919 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1462/2934] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:30:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:30:29.384573 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb3b9e0-833e-44d1-9001-aa90996ee161 service neutron] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:30:29.408472 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb3b9e0-833e-44d1-9001-aa90996ee161 service neutron] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:30:29.418705 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b32ed1-e48a-451a-b5ef-7d3484d73e2b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f57872de7bb74ea48d30e93e51830aba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:30:29.420405 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b32ed1-e48a-451a-b5ef-7d3484d73e2b service placement] REQUEST_METHOD: `GET` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:30:29.420918 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b32ed1-e48a-451a-b5ef-7d3484d73e2b service placement] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:30:29.421847 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b32ed1-e48a-451a-b5ef-7d3484d73e2b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:30:29.425028 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-55b32ed1-e48a-451a-b5ef-7d3484d73e2b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:30:29.449004 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b32ed1-e48a-451a-b5ef-7d3484d73e2b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:30:29.452778 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b32ed1-e48a-451a-b5ef-7d3484d73e2b service placement] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:30:29.484665 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1473/2935] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:30:29 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:30:29.521284 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee79bddd-5720-40f0-b651-de1ae9e8979e service placement] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:30:29.523255 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-ee79bddd-5720-40f0-b651-de1ae9e8979e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:30:29.550247 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee79bddd-5720-40f0-b651-de1ae9e8979e service placement] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:30:29.564258 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd24f37-e28a-4bf0-aeff-180526ebc182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:30:29.565350 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd24f37-e28a-4bf0-aeff-180526ebc182 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:30:29.565759 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd24f37-e28a-4bf0-aeff-180526ebc182 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:30:29.568232 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd24f37-e28a-4bf0-aeff-180526ebc182 admin admin] PATH_INFO: `/v3/users/7f661aa1f00e4c7e916b2498505475dc` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:30:29.568232 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd24f37-e28a-4bf0-aeff-180526ebc182 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f661aa1f00e4c7e916b2498505475dc)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:30:29.569390 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd24f37-e28a-4bf0-aeff-180526ebc182 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:30:29.652586 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-4cd24f37-e28a-4bf0-aeff-180526ebc182 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f661aa1f00e4c7e916b2498505475dc'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:30:29.663817 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-4cd24f37-e28a-4bf0-aeff-180526ebc182 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f661aa1f00e4c7e916b2498505475dc'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:30:29.664754 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-4cd24f37-e28a-4bf0-aeff-180526ebc182 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f661aa1f00e4c7e916b2498505475dc'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:30:29.665577 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-4cd24f37-e28a-4bf0-aeff-180526ebc182 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f661aa1f00e4c7e916b2498505475dc'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:30:29.677102 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-4cd24f37-e28a-4bf0-aeff-180526ebc182 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f661aa1f00e4c7e916b2498505475dc'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:30:29.700224 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1463/2936] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:30:29 2022] DELETE /identity/v3/users/7f661aa1f00e4c7e916b2498505475dc => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:30:29.945978 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b32ed1-e48a-451a-b5ef-7d3484d73e2b service placement] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:30:29.966848 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b32ed1-e48a-451a-b5ef-7d3484d73e2b service placement] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:30:30.057522 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3972f517-3b55-42d9-8097-c7d4f3fd07c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:30:30.058300 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3972f517-3b55-42d9-8097-c7d4f3fd07c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:30:30.058644 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3972f517-3b55-42d9-8097-c7d4f3fd07c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:30:30.058959 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3972f517-3b55-42d9-8097-c7d4f3fd07c5 admin admin] PATH_INFO: `/v3/projects/ed4697b9e0a940b284b392b3e93197c7` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:30:30.060661 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3972f517-3b55-42d9-8097-c7d4f3fd07c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed4697b9e0a940b284b392b3e93197c7)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:30:30.062883 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3972f517-3b55-42d9-8097-c7d4f3fd07c5 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:30:30.129170 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-3972f517-3b55-42d9-8097-c7d4f3fd07c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed4697b9e0a940b284b392b3e93197c7', 'request_id': 'req-3972f517-3b55-42d9-8097-c7d4f3fd07c5'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:30:30.139814 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-3972f517-3b55-42d9-8097-c7d4f3fd07c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed4697b9e0a940b284b392b3e93197c7', 'request_id': 'req-3972f517-3b55-42d9-8097-c7d4f3fd07c5'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:30:30.151328 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-3972f517-3b55-42d9-8097-c7d4f3fd07c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed4697b9e0a940b284b392b3e93197c7', 'request_id': 'req-3972f517-3b55-42d9-8097-c7d4f3fd07c5'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:30:30.171624 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-3972f517-3b55-42d9-8097-c7d4f3fd07c5 admin admin] The token cache is being invalidate because project ed4697b9e0a940b284b392b3e93197c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81772) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:30:30.172119 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-3972f517-3b55-42d9-8097-c7d4f3fd07c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:30:30.173235 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-3972f517-3b55-42d9-8097-c7d4f3fd07c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:30:30.175935 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1474/2937] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:30:29 2022] DELETE /identity/v3/projects/ed4697b9e0a940b284b392b3e93197c7 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:30:30.181702 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd24f37-e28a-4bf0-aeff-180526ebc182 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:30:30.183975 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-4cd24f37-e28a-4bf0-aeff-180526ebc182 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:30:30.207837 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd24f37-e28a-4bf0-aeff-180526ebc182 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:30:30.304393 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ec3e18-12f0-4b32-870a-549afc7c22c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:30:30.305322 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ec3e18-12f0-4b32-870a-549afc7c22c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:30:30.305772 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ec3e18-12f0-4b32-870a-549afc7c22c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:30:30.306104 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ec3e18-12f0-4b32-870a-549afc7c22c9 admin admin] PATH_INFO: `/v3/users/b2219ed125b2449f8a738834af8808b5` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:30:30.307679 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ec3e18-12f0-4b32-870a-549afc7c22c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2219ed125b2449f8a738834af8808b5)` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:30:30.310245 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ec3e18-12f0-4b32-870a-549afc7c22c9 admin admin] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:30:30.372054 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-42ec3e18-12f0-4b32-870a-549afc7c22c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2219ed125b2449f8a738834af8808b5'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:30:30.374565 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:30:30.384490 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-42ec3e18-12f0-4b32-870a-549afc7c22c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2219ed125b2449f8a738834af8808b5'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:30:30.386347 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-42ec3e18-12f0-4b32-870a-549afc7c22c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2219ed125b2449f8a738834af8808b5'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:30:30.387709 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-42ec3e18-12f0-4b32-870a-549afc7c22c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2219ed125b2449f8a738834af8808b5'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:30:30.397774 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.notifications [None req-42ec3e18-12f0-4b32-870a-549afc7c22c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2219ed125b2449f8a738834af8808b5'} {{(pid=81771) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:30:30.421310 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1464/2938] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 6 20:30:30 2022] DELETE /identity/v3/users/b2219ed125b2449f8a738834af8808b5 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:30:30.658829 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3972f517-3b55-42d9-8097-c7d4f3fd07c5 admin admin] Authenticating user token {{(pid=81772) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:30:30.661703 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.fernet_utils [None req-3972f517-3b55-42d9-8097-c7d4f3fd07c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81772) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:30:30.687275 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3972f517-3b55-42d9-8097-c7d4f3fd07c5 admin admin] Validating token access rules against request {{(pid=81772) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:30:30.703406 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:30:30.783289 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2401eba4-e3a6-43cb-af04-a65c86186b41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e7095a0137b45c984179dcf25de9362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a630b7748ecd4893b8db1b6c1dc4efe9', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81772) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:30:30.784111 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2401eba4-e3a6-43cb-af04-a65c86186b41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:30:30.784528 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2401eba4-e3a6-43cb-af04-a65c86186b41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:30:30.784925 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2401eba4-e3a6-43cb-af04-a65c86186b41 admin admin] PATH_INFO: `/v3/projects/f3f976e76eb841ea871010a2eef1dc8f` {{(pid=81772) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:30:30.786716 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2401eba4-e3a6-43cb-af04-a65c86186b41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3f976e76eb841ea871010a2eef1dc8f)` {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:30:30.789098 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2401eba4-e3a6-43cb-af04-a65c86186b41 admin admin] RBAC: Authorization granted {{(pid=81772) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:30:30.852971 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-2401eba4-e3a6-43cb-af04-a65c86186b41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3f976e76eb841ea871010a2eef1dc8f', 'request_id': 'req-2401eba4-e3a6-43cb-af04-a65c86186b41'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:30:30.865080 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-2401eba4-e3a6-43cb-af04-a65c86186b41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3f976e76eb841ea871010a2eef1dc8f', 'request_id': 'req-2401eba4-e3a6-43cb-af04-a65c86186b41'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:30:30.876068 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-2401eba4-e3a6-43cb-af04-a65c86186b41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3f976e76eb841ea871010a2eef1dc8f', 'request_id': 'req-2401eba4-e3a6-43cb-af04-a65c86186b41'} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:30:30.894709 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-2401eba4-e3a6-43cb-af04-a65c86186b41 admin admin] The token cache is being invalidate because project f3f976e76eb841ea871010a2eef1dc8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81772) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 20:30:30.895174 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-2401eba4-e3a6-43cb-af04-a65c86186b41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:30:30.896279 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG keystone.notifications [None req-2401eba4-e3a6-43cb-af04-a65c86186b41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81772) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 20:30:30.899053 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: [pid: 81772|app: 0|req: 1475/2939] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 6 20:30:30 2022] DELETE /identity/v3/projects/f3f976e76eb841ea871010a2eef1dc8f => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 20:30:40.416486 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=12 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:30:40.889471 dv-cinder-847732-4-iscsi devstack@keystone.service[81772]: DEBUG dbcounter [-] [81772] Writing DB stats keystone:SELECT=46,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81772) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 20:34:04.010616 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ec3e18-12f0-4b32-870a-549afc7c22c9 admin admin] Authenticating user token {{(pid=81771) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 20:34:04.012444 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-42ec3e18-12f0-4b32-870a-549afc7c22c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:34:04.210311 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ec3e18-12f0-4b32-870a-549afc7c22c9 admin admin] Validating token access rules against request {{(pid=81771) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 20:34:04.286256 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019b9cc3-8b98-4647-81c9-0f6a4cca349c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f57872de7bb74ea48d30e93e51830aba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6aef43493bb1420a9e122313d8db7d5a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81771) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 20:34:04.287196 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019b9cc3-8b98-4647-81c9-0f6a4cca349c service placement] REQUEST_METHOD: `GET` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 20:34:04.287671 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019b9cc3-8b98-4647-81c9-0f6a4cca349c service placement] SCRIPT_NAME: `/identity` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 20:34:04.288097 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019b9cc3-8b98-4647-81c9-0f6a4cca349c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81771) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 20:34:04.290415 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.fernet_utils [None req-019b9cc3-8b98-4647-81c9-0f6a4cca349c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81771) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 20:34:04.306506 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019b9cc3-8b98-4647-81c9-0f6a4cca349c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 20:34:04.308313 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019b9cc3-8b98-4647-81c9-0f6a4cca349c service placement] RBAC: Authorization granted {{(pid=81771) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 20:34:04.322323 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: [pid: 81771|app: 0|req: 1465/2940] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 6 20:34:04 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 20:34:14.316895 dv-cinder-847732-4-iscsi devstack@keystone.service[81771]: DEBUG dbcounter [-] [81771] Writing DB stats keystone:SELECT=89 {{(pid=81771) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}